were used to generate four sets of pseudo-labels and subsequently to train the
final 3D segmentation network. Results showed that this slice-based strategy
outperforms uniform sampling. Building upon this method, Wu et al. [2022c]
incorporated a self-attention module into the autoencoder to enhance slice-
level feature learning. In recent years, uncertainty methods have been
introduced for selecting key slices. In interactive segmentation of 3D medical
images, Zhou et al. [2021a] and their subsequent work [Zhou et al., 2022]
introduced a quality assessment module to provide a predicted average IoU
score for each slice. They chose the slice with the lowest IoU score in each
volume for the next round of interactive segmentation. In muscle segmentation
of CT images, Hiasa et al. [2020] selected key slices and key regions. This
work adopted clustering to select key slices and further selected regions with
high uncertainty within each key slice for annotation.

####  5.2.2 One-shot Annotation

Currently, most AL works require multiple rounds of annotation. However, this
setting may not be practical in medical image segmentation. Multi-round
annotation requires physicians to be readily available for each round of
labeling, which is unrealistic in practice. If physicians cannot complete the
annotations on time, the AL process must be suspended. In contrast, one-shot
annotation eliminates the need for multiple interactions with physicians. It
also allows for selecting valuable samples in a single round, thus reducing
time costs. Both one-shot annotation and cold-start AL aim to select the most
optimal initial annotations. However, the former allows for a higher
annotation budget and strictly limits the number of interactions with experts
to just one. Most relevant works combine self-supervised features and specific
sampling strategies to achieve one-shot annotation. For example, RA [Zheng et
al., 2019] is one of the earliest works in one-shot AL for medical image
segmentation. They applied the VAE feature and a representativeness strategy
to select informative samples for annotation in one shot. RA performed
excellently in segmenting gland of pathological images, whole-heart of MRI
images, and fungal of electron microscopic images. Jin et al. [2022b] combined
features of contrastive learning with farthest-first sampling to achieve one-
shot annotation. The proposed method demonstrated effectiveness on the ISIC
2018 and lung segmentation datasets. Additionally, Jin et al. [2023b] utilized
auto-encoding transformations for self-supervised feature learning. They
selected and annotated samples with high density based on reachable distance.

####  5.2.3 Annotation Cost

Current AL works often assume equal annotation costs for each sample. Yet,
this is not the case in medical image segmentation, where the time to annotate
different samples can differ greatly. AL techniques can better support
physicians by considering annotation costs (e.g., annotation time).
Nevertheless, there is still limited research in this specific domain. In
detecting intracranial hemorrhage of CT scans, Kuo et al. [2018] combined
predictive disagreement with annotation time to select samples for annotation.
Specifically, they adopted the Jensen-Shannon divergence to measure the
disagreement between the outputs of multiple models. Annotation time for each
sample was estimated by the length of the segmentation boundary and the number
of connected components. In this work, AL was framed as a 0-1 knapsack
problem, and dynamic programming is used to solve this problem for selecting
informative samples. In brain structure segmentation, Atzeni et al. [2022]
further considered the spatial relationships between multiple regions of
interest to more accurately estimate the annotation cost. Moreover, the
average Dice coefficient of previous rounds was used to predict the average
Dice for current segmentation results. They selected and annotated regions
that can maximize the average Dice.

###  5.3 Active Learning in Medical Image Reconstruction

AL can also be applied in medical image reconstruction. AL methods can help
minimize the observations needed for modalities that require a long imaging
time. This accelerates the imaging process and shortens the waiting period for
patients. In this section, we’ll explore the application of AL in the
reconstruction of MRI, CT, and electron microscopy.

Deep learning has been applied to accelerate MRI acquisition and
reconstruction. A common practice is to reduce k-space sampling through a
fixed mask and use a deep model to reconstruct the undersampled MRI [Qin et
al., 2018]. To further improve the imaging speed, learnable sampling in AL can
be applied to select the next measurement locations in k-space. For example,
Zhang et al. [2019] adopted adversarial learning to train an evaluator for
selecting the next row in k-space. Pineda et al. [2020] utilized reinforcement
learning to train a dual deep Q-network for active sampling in k-space. Bakker
et al. [2020] adopted policy gradient in reinforcement learning to train a
policy network for adaptive sampling in k-space. The reward for the policy
network was based on the improvement in structural similarity before and after
the acquisition. Additionally, Bakker et al. [2022] explored how to jointly
optimize the reconstruction and acquisition networks.

In addition to MRI imaging, AL has been employed in CT reconstruction as
illustrated by Wang et al. [2022a]. They adaptively chose the scanning angles
tailored to individual patients, leading to a reduction in both radiation
exposure and scanning duration. In electron microscopy, Mi et al. [2020]
initially enhanced low-resolution images to high-resolution and then predicted
the location of region-of-interest and reconstruction error. A weighted DPP
based on reconstruction error was applied to select pixels that needed
rescanned. Results showed that weighted DPP maintained both low reconstruction
error and spatial diversity.

##  6 Challenges and Future Perspectives

Currently, annotation scarcity is a significant bottleneck hindering the
development of medical image analysis. AL improves annotation efficiency by
selectively querying the most informative samples for annotation. This survey
reviews the recent developments in deep active learning, focusing on the
evaluation of informativeness, sampling strategies, integration with other
label-efficient techniques, and the application of AL in medical image
analysis. In this section, we will discuss the existing challenges faced by AL
in medical image analysis and its future perspectives.

###  6.1 Towards Active Learning with Better Uncertainty

In AL, uncertainty plays a pivotal role. However, it would be beneficial if
the uncertainty more directly highlighted the model’s mistakes. We can enhance
the model’s performance by querying samples with inaccurate predictions.

Recently, many works have adopted learnable performance estimation for quality
control of deep model outputs. For instance, the recently proposed segment
anything model (SAM) [Kirillov et al., 2023] provides IoU estimates for each
mask to evaluate its quality. In medical image analysis, automated quality
control is critical to ensure the reliability and safety of the deep model
outputs [Kohlberger et al., 2012]. For example, Wang et al. [2020c] employed
deep generative models for learnable quality control in cardiac MRI
segmentation, where the predicted Dice scores showed a strong linear
relationship with the real ones. Additionally, Billot et al. [2023] used an
additional neural network to predict the Dice coefficient of brain tissue
segmentation results. Overall, learnable performance estimation can accurately
predict the quality of model outputs. Hence, delving deeper into their
potential for uncertainty-based AL is crucial to effectively tackle the issue
of over-confidence.

Moreover, improving the probability calibration of model prediction is a
promising way to mitigate the over-confidence issue. Calibration [Guo et al.,
2017, Mehrtash et al., 2020] reflects the consistency between model prediction
probabilities and the ground truth. A well-calibrated model should display a
strong correlation between confidence and accuracy. For instance, if a
perfect-calibrated polyp classifier gives an average confidence score of 0.9
on a dataset, it means that 90% of those samples should indeed have polyps. In
reality, deep models generally suffer from the issue of over-confidence, which
essentially means that they are not well-calibrated. Currently, only a few
uncertainty-based AL works have considered probability calibration. For
instance, Beluch et al. [2018] found that the model ensemble has better
calibration than MC Dropout. Xie et al. [2022c] mitigated miscalibration by
considering all possible prediction outcomes in the Dirichlet distribution.
However, these methods are limited to proposing a better uncertainty metric
and validating the calibration quality post-hoc. Existing calibration methods
[Guo et al., 2017, Ding et al., 2021] directly adjusted the distribution of
prediction probabilities. However, these methods require an additional labeled
dataset, thus limiting their practical applicability. Therefore, integrating
probability calibration into uncertainty-based AL represents a valuable
research direction worth exploring.

###  6.2 Towards Active Learning with Better Representativeness

Representativeness-based AL effectively utilizes feature representations and
data distributions for sample selection. Cover-based and diversity-based AL
methods implicitly capture the data distribution, whereas density-based AL
explicitly estimates it. However, the latter requires supplementary strategies
to ensure diversity. As the core of density-based AL, density estimation in
high-dimensional spaces has always been challenging. Popular density
estimation methods, such as kernel density estimation and GMM, may face
challenges in high-dimensional spaces. In future research, we can consider
introducing density estimators tailored to high-dimensional spaces.

###  6.3 Towards Active Learning with Weak Annotation

In §4.4, we discuss region-based active learning, which only requires region-
level annotation of a sample. However, annotating all pixels within the region
is still needed. Some AL works have incorporated weak annotations to simplify
the task for annotators. In object detection tasks, Vo et al. [2022] trained
deep models with image-level annotation. They selected samples with box-in-box
prediction results and annotated them with bounding boxes. Moreover, Lyu et
al. [2023] adopted disagreement to choose which objects are worth annotating.
Rather than annotating all objects within the image, they only required box-
level annotations for a subset of objects. In AL of instance segmentation,
Tang et al. [2022] only required annotations for each object’s class label and
bounding box, without the annotation of fine-grained segmentation masks. In
future research, AL based on weak annotations is a direction worthy of in-
depth exploration.

###  6.4 Towards Active Learning with Better Generative Models

In §4.5, we summarize the applications of generative models in AL. However,
existing works have mainly focused on using GANs as sample generators.
Recently, diffusion models [Kazerouni et al., 2023] have advanced in achieving
state-of-the-art generative quality. Furthermore, text-to-image diffusion
models, represented by Stable Diffusion [Rombach et al., 2022], have
revolutionized the image generation domain. Their high-quality, text-guided
generation results enable a more flexible image generation. Exploring the
potential of diffusion models in deep AL is a promising avenue for future
research.

###  6.5 Towards Active Learning with Foundation Models

With the rise of visual foundational models, such as contrastive language-
image pretraining (CLIP) [Radford et al., 2021] and SAM [Kirillov et al.,
2023], and large language models (LLMs) like GPT [OpenAI, 2023], deep learning
in medical image analysis and computer vision is undergoing a paradigm shift.
These foundational models [Bommasani et al., 2021] offer new opportunities for
the development of AL.

AL is closely related to the training paradigms in deep learning of computer
vision and medical image analysis. From the initial approach of train-from-
scratch to the “pretrain-finetune” strategy using supervised or self-
supervised pre-trained models, these paradigms usually require fine-tuning the
entire network. Foundation models contain a wealth of knowledge. When combined
with recently emerging parameter-efficient fine tuning (PEFT) or prompt tuning
techniques [Hu et al., 2021, Jia et al., 2022], we can tune only a minimal
subset of model weights (for example, 5%) for rapid transfer to downstream
tasks. As the number of fine-tuned parameters decreases, AL has the potential
to further reduce the number of required annotated samples. Therefore, it is
essential to investigate the applicability of existing AL under PEFT or prompt
tuning and explore the most suitable AL strategies for PEFT.

In natural language processing, LLMs have already taken a dominant role. Since
most researchers cannot tune the LLMs, they rely on in-context learning, which
provides LLMs with limited examples to transfer to downstream tasks. We
believe that visual in-context learning will play a vital role in future
research. Therefore, selecting the most suitable prompts for visual in-context
learning will become an important research direction of AL.

##  7 Conclusion

Active learning is important to deep learning in medical image analysis since
it effectively reduces the annotation costs incurred by human experts. This
survey comprehensively reviews the core methods in deep active learning, its
integration with different label-efficient techniques, and active learning
works tailored to medical image analysis. We further discuss its current
challenges and future perspectives. In summary, we believe that deep active
learning and its application in medical image analysis hold important academic
value and clinical potential, with ample room for further development.

## Acknowledgments

This study was supported by the National Natural Science Foundation of China
(Grant 82372097 and 82072021) and the Science and Technology Innovation Plan
of Shanghai Science and Technology Commission (Grant 23S41900400).

## References

  * Achanta et al. [2012] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S., 2012\.  Slic superpixels compared to state-of-the-art superpixel methods.  IEEE transactions on pattern analysis and machine intelligence 34, 2274–2282.
  * Agarwal et al. [2020] Agarwal, S., Arora, H., Anand, S., Arora, C., 2020\.  Contextual diversity for active learning, in: Computer Vision – ECCV 2020. Springer International Publishing, Cham. volume 12361, pp. 137–153.  doi:10.1007/978-3-030-58517-4_9.
  * Aghdam et al. [2019] Aghdam, H.H., Gonzalez-Garcia, A., Weijer, J.v.d., López, A.M., 2019\.  Active learning for deep detection neural networks, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3672–3680.
  * Angluin [1988] Angluin, D., 1988.  Queries and concept learning.  Machine Learning 2, 319–342.  doi:10.1023/A:1022821128753.
  * Angluin [2004] Angluin, D., 2004.  Queries revisited.  Theoretical Computer Science 313, 175–194.  doi:10.1016/j.tcs.2003.11.004.
  * Ardila et al. [2019] Ardila, D., Kiraly, A.P., Bharadwaj, S., Choi, B., Reicher, J.J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., et al., 2019\.  End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography.  Nature medicine 25, 954–961.
  * Ash et al. [2021] Ash, J., Goel, S., Krishnamurthy, A., Kakade, S., 2021\.  Gone fishing: Neural active learning with fisher embeddings, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 8927–8939.
  * Ash et al. [2020] Ash, J.T., Zhang, C., Krishnamurthy, A., Langford, J., Agarwal, A., 2020.  Deep batch active learning by diverse, uncertain gradient lower bounds, in: International Conference on Learning Representations.
  * Atzeni et al. [2022] Atzeni, A., Peter, L., Robinson, E., Blackburn, E., Althonayan, J., Alexander, D.C., Iglesias, J.E., 2022.  Deep active learning for suggestive segmentation of biomedical image stacks via optimisation of dice scores and traced boundary length.  Medical Image Analysis 81, 102549\.  doi:10.1016/j.media.2022.102549.
  * Bai et al. [2022] Bai, F., Xing, X., Shen, Y., Ma, H., Meng, M.Q.H., 2022\.  Discrepancy-based active learning for weakly supervised bleeding segmentation in wireless capsule endoscopy images, in: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Springer Nature Switzerland, Cham. pp. 24–34.  doi:10.1007/978-3-031-16452-1_3.
  * Baid et al. [2021] Baid, U., Ghodasara, S., Mohan, S., Bilello, M., Calabrese, E., Colak, E., Farahani, K., Kalpathy-Cramer, J., Kitamura, F.C., Pati, S., et al., 2021\.  The rsna-asnr-miccai brats 2021 benchmark on brain tumor segmentation and radiogenomic classification.  arXiv preprint arXiv:2107.02314 .
  * Bakker et al. [2022] Bakker, T., Muckley, M., Romero-Soriano, A., Drozdzal, M., Pineda, L., 2022.  On learning adaptive acquisition policies for undersampled multi-coil mri reconstruction, in: Proceedings of The 5th International Conference on Medical Imaging with Deep Learning, PMLR. pp. 63–85.
  * Bakker et al. [2020] Bakker, T., van Hoof, H., Welling, M., 2020.  Experimental design for mri by greedy policy search, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 18954–18966.
  * Balaram et al. [2022] Balaram, S., Nguyen, C.M., Kassim, A., Krishnaswamy, P., 2022\.  Consistency-based semi-supervised evidential active learning for diagnostic radiograph classification, in: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Springer Nature Switzerland, Cham. pp. 675–685.  doi:10.1007/978-3-031-16431-6_64.
  * Baltruschat et al. [2019] Baltruschat, I.M., Nickisch, H., Grass, M., Knopp, T., Saalbach, A., 2019.  Comparison of deep learning approaches for multi-label chest x-ray classification.  Scientific reports 9, 6381\.
  * Beluch et al. [2018] Beluch, W.H., Genewein, T., Nürnberger, A., Köhler, J.M., 2018\.  The power of ensembles for active learning in image classification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9368–9377.
  * Bengar et al. [2022] Bengar, J.Z., van de Weijer, J., Fuentes, L.L., Raducanu, B., 2022\.  Class-balanced active learning for image classification, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1536–1545.
  * Bengar et al. [2021] Bengar, J.Z., van de Weijer, J., Twardowski, B., Raducanu, B., 2021\.  Reducing label effort: Self-supervised meets active learning, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1631–1639.
  * Van den Bergh et al. [2012] Van den Bergh, M., Boix, X., Roig, G., De Capitani, B., Van Gool, L., 2012.  Seeds: Superpixels extracted via energy-driven sampling, in: Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VII 12, Springer. pp. 13–26.
  * Billot et al. [2023] Billot, B., Magdamo, C., Cheng, Y., Arnold, S.E., Das, S., Iglesias, J.E., 2023\.  Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain mri datasets.  Proceedings of the National Academy of Sciences 120, e2216399120.
  * Bishop [1994] Bishop, C.M., 1994.  Mixture density networks .
  * Bıyık et al. [2019] Bıyık, E., Wang, K., Anari, N., Sadigh, D., 2019\.  Batch active learning using determinantal point processes.  arXiv preprint arXiv:1906.07975 .
  * Bommasani et al. [2021] Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., et al., 2021\.  On the opportunities and risks of foundation models.  arXiv preprint arXiv:2108.07258 .
  * Budd et al. [2021] Budd, S., Robinson, E.C., Kainz, B., 2021.  A survey on active learning and human-in-the-loop deep learning for medical image analysis.  Medical Image Analysis 71, 102062\.
  * Cai et al. [2021] Cai, L., Xu, X., Liew, J.H., Foo, C.S., 2021.  Revisiting superpixels for active learning in semantic segmentation with realistic annotation costs, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10988–10997.
  * Caramalau et al. [2021] Caramalau, R., Bhattarai, B., Kim, T.K., 2021.  Sequential graph convolutional network for active learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9583–9592.
  * Cardoso et al. [2017] Cardoso, T.N.C., Silva, R.M., Canuto, S., Moro, M.M., Gonçalves, M.A., 2017.  Ranked batch-mode active learning.  Information Sciences 379, 313–337.  doi:10.1016/j.ins.2016.10.037.
  * Casanova et al. [2020] Casanova, A., Pinheiro, P.O., Rostamzadeh, N., Pal, C.J., 2020\.  Reinforced active learning for image segmentation, in: International Conference on Learning Representations.
  * Chaudhuri et al. [2015] Chaudhuri, K., Kakade, S.M., Netrapalli, P., Sanghavi, S., 2015\.  Convergence rates of active learning for maximum likelihood estimation, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.
  * Chen et al. [2021] Chen, J., Xie, Y., Wang, K., Zhang, C., Vannan, M.A., Wang, B., Qian, Z., 2021.  Active image synthesis for efficient labeling.  IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 3770–3781.  doi:10.1109/TPAMI.2020.2993221.
  * Chen et al. [2023] Chen, L., Bai, Y., Huang, S., Lu, Y., Wen, B., Yuille, A.L., Zhou, Z., 2023\.  Making your first choice: To address cold start problem in vision active learning, in: Medical Imaging with Deep Learning.
  * Chen et al. [2020] Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.E., 2020.  Big self-supervised models are strong semi-supervised learners.  Advances in neural information processing systems 33, 22243–22255.
  * Chen et al. [2022a] Chen, Y., Mancini, M., Zhu, X., Akata, Z., 2022a.  Semi-supervised and unsupervised deep visual learning: A survey.  IEEE transactions on pattern analysis and machine intelligence .
  * Chen et al. [2022b] Chen, Z., Zhang, J., Wang, P., Chen, J., Li, J., 2022b.  When active learning meets implicit semantic data augmentation, in: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (Eds.), Computer Vision – ECCV 2022. Springer Nature Switzerland, Cham. volume 13685, pp. 56–72.  doi:10.1007/978-3-031-19806-9_4.
  * Choi et al. [2021a] Choi, J., Elezi, I., Lee, H.J., Farabet, C., Alvarez, J.M., 2021a.  Active learning for deep object detection via probabilistic modeling, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10264–10273.
  * Choi et al. [2021b] Choi, J., Yi, K.M., Kim, J., Choo, J., Kim, B., Chang, J., Gwon, Y., Chang, H.J., 2021b.  Vab-al: Incorporating class imbalance and difficulty with variational bayes for active learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6749–6758.
  * Citovsky et al. [2021] Citovsky, G., DeSalvo, G., Gentile, C., Karydas, L., Rajagopalan, A., Rostamizadeh, A., Kumar, S., 2021.  Batch active learning at scale, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 11933–11944.
  * Cohn et al. [1994] Cohn, D., Atlas, L., Ladner, R., 1994.  Improving generalization with active learning.  Machine Learning 15, 201–221.  doi:10.1007/BF00993277.
  * Dai et al. [2022] Dai, C., Wang, S., Mo, Y., Angelini, E., Guo, Y., Bai, W., 2022.  Suggestive annotation of brain mr images with gradient-guided sampling.  Medical Image Analysis 77, 102373\.  doi:10.1016/j.media.2022.102373.
  * Dai et al. [2020] Dai, C., Wang, S., Mo, Y., Zhou, K., Angelini, E., Guo, Y., Bai, W., 2020.  Suggestive annotation of brain tumour images with gradient-guided sampling, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Springer International Publishing, Cham. volume 12264, pp. 156–165.  doi:10.1007/978-3-030-59719-1_16.
  * Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L., 2009\.  Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee. pp. 248–255.
  * Ding et al. [2021] Ding, Z., Han, X., Liu, P., Niethammer, M., 2021.  Local temperature scaling for probability calibration, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6889–6899.
  * Du et al. [2022] Du, P., Chen, H., Zhao, S., Chai, S., Chen, H., Li, C., 2022.  Contrastive active learning under class distribution mismatch.  IEEE Transactions on Pattern Analysis and Machine Intelligence , 1–13doi:10.1109/TPAMI.2022.3188807.
  * Du et al. [2021] Du, P., Zhao, S., Chen, H., Chai, S., Chen, H., Li, C., 2021.  Contrastive coding for active learning under class distribution mismatch, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8927–8936.
  * Ducoffe and Precioso [2018] Ducoffe, M., Precioso, F., 2018\.  Adversarial active learning for deep networks: a margin based approach.  arXiv preprint arXiv:1802.09841 .
  * Esteva et al. [2017] Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S., 2017.  Dermatologist-level classification of skin cancer with deep neural networks.  nature 542, 115–118.
  * Farahani and Hekmatfar [2009] Farahani, R.Z., Hekmatfar, M., 2009\.  Facility location: concepts, models, algorithms and case studies.  Springer Science & Business Media.
  * Feige [1998] Feige, U., 1998.  A threshold of ln n for approximating set cover.  Journal of the ACM (JACM) 45, 634–652.
  * Fu et al. [2021] Fu, B., Cao, Z., Wang, J., Long, M., 2021.  Transferable query selection for active domain adaptation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7272–7281.
  * Fujishige [2005] Fujishige, S., 2005.  Submodular functions and optimization.  Elsevier.
  * Gal and Ghahramani [2016] Gal, Y., Ghahramani, Z., 2016\.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning, in: international conference on machine learning, PMLR. pp. 1050–1059.
  * Gal et al. [2017] Gal, Y., Islam, R., Ghahramani, Z., 2017.  Deep bayesian active learning with image data, in: Proceedings of the 34th International Conference on Machine Learning, PMLR. pp. 1183–1192.
  * Gao et al. [2020] Gao, M., Zhang, Z., Yu, G., Arık, S.Ö., Davis, L.S., Pfister, T., 2020.  Consistency-based semi-supervised active learning: Towards minimizing labeling cost, in: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (Eds.), Computer Vision – ECCV 2020. Springer International Publishing, Cham. volume 12355, pp. 510–526.  doi:10.1007/978-3-030-58607-2_30.
  * Gidaris et al. [2018] Gidaris, S., Singh, P., Komodakis, N., 2018.  Unsupervised representation learning by predicting image rotations, in: International Conference on Learning Representations.
  * Gissin and Shalev-Shwartz [2019] Gissin, D., Shalev-Shwartz, S., 2019\.  Discriminative active learning.  arXiv preprint arXiv:1907.06347 .
  * Gong et al. [2022] Gong, J., Fan, Z., Ke, Q., Rahmani, H., Liu, J., 2022\.  Meta agent teaming active learning for pose estimation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11079–11089.
  * Goodfellow et al. [2014a] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2014a.  Generative adversarial nets.  Advances in neural information processing systems 27\.
  * Goodfellow et al. [2014b] Goodfellow, I.J., Shlens, J., Szegedy, C., 2014b.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572 .
  * Guan and Liu [2021] Guan, H., Liu, M., 2021.  Domain adaptation for medical image analysis: a survey.  IEEE Transactions on Biomedical Engineering 69, 1173–1185.
  * Guo et al. [2017] Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q., 2017.  On calibration of modern neural networks, in: International conference on machine learning, PMLR. pp. 1321–1330.
  * Hacohen et al. [2022] Hacohen, G., Dekel, A., Weinshall, D., 2022.  Active learning on a budget: Opposite strategies suit high and low budgets, in: International Conference on Machine Learning, PMLR. pp. 8175–8195.
  * Haußmann et al. [2019] Haußmann, M., Hamprecht, F., Kandemir, M., 2019.  Deep active learning with adaptive acquisition, in: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 2470–2476.
  * He et al. [2022] He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R., 2022.  Masked autoencoders are scalable vision learners, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000–16009.
  * He et al. [2020] He, K., Fan, H., Wu, Y., Xie, S., Girshick, R., 2020\.  Momentum contrast for unsupervised visual representation learning, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729–9738.
  * Heo et al. [2019] Heo, B., Lee, M., Yun, S., Choi, J.Y., 2019.  Knowledge distillation with adversarial samples supporting decision boundary, in: Proceedings of the AAAI conference on artificial intelligence, pp. 3771–3778.
  * Hiasa et al. [2020] Hiasa, Y., Otake, Y., Takao, M., Ogawa, T., Sugano, N., Sato, Y., 2020\.  Automated muscle segmentation from clinical ct using bayesian u-net for personalized musculoskeletal modeling.  IEEE Transactions on Medical Imaging 39, 1030–1040.  doi:10.1109/TMI.2019.2940555.
  * Hochbaum and Shmoys [1985] Hochbaum, D.S., Shmoys, D.B., 1985\.  A best possible heuristic for the k-center problem.  Mathematics of operations research 10, 180–184.
  * Hou et al. [2021] Hou, J., Graham, B., Niessner, M., Xie, S., 2021\.  Exploring data-efficient 3d scene understanding with contrastive scene contexts, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15587–15597.
  * Houlsby et al. [2011] Houlsby, N., Huszár, F., Ghahramani, Z., Lengyel, M., 2011\.  Bayesian active learning for classification and preference learning.  arXiv preprint arXiv:1112.5745 .
  * Hu et al. [2021] Hu, E.J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al., 2021.  Lora: Low-rank adaptation of large language models, in: International Conference on Learning Representations.
  * Hu et al. [2022] Hu, Z., Bai, X., Zhang, R., Wang, X., Sun, G., Fu, H., Tai, C.L., 2022.  Lidal: Inter-frame uncertainty based active learning for 3d lidar semantic segmentation, in: European Conference on Computer Vision, Springer. pp. 248–265.
  * Huang et al. [2023] Huang, D., Li, J., Chen, W., Huang, J., Chai, Z., Li, G., 2023.  Divide and adapt: Active domain adaptation via customized learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7651–7660.
  * Huang et al. [2017] Huang, G., Li, Y., Pleiss, G., Liu, Z., Hopcroft, J.E., Weinberger, K.Q., 2017.  Snapshot ensembles: Train 1, get m for free.  arXiv preprint arXiv:1704.00109 .
  * Huang et al. [2021] Huang, S., Wang, T., Xiong, H., Huan, J., Dou, D., 2021.  Semi-supervised active learning with temporal output discrepancy, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3447–3456.
  * Huang et al. [2020] Huang, Y.J., Liu, W., Wang, X., Fang, Q., Wang, R., Wang, Y., Chen, H., Chen, H., Meng, D., Wang, L., 2020.  Rectifying supporting regions with mixed and active supervision for rib fracture recognition.  IEEE Transactions on Medical Imaging 39, 3843–3854.  doi:10.1109/TMI.2020.3006138.
  * Hwang et al. [2022] Hwang, S., Lee, S., Kim, S., Ok, J., Kwak, S., 2022\.  Combating label distribution shift for active domain adaptation, in: European Conference on Computer Vision, Springer. pp. 549–566.
  * Jaeger et al. [2013] Jaeger, S., Karargyris, A., Candemir, S., Folio, L., Siegelman, J., Callaghan, F., Xue, Z., Palaniappan, K., Singh, R.K., Antani, S., et al., 2013\.  Automatic tuberculosis screening using chest radiographs.  IEEE transactions on medical imaging 33, 233–245.
  * Jia et al. [2022] Jia, M., Tang, L., Chen, B.C., Cardie, C., Belongie, S., Hariharan, B., Lim, S.N., 2022\.  Visual prompt tuning, in: European Conference on Computer Vision, Springer. pp. 709–727.
  * Jin et al. [2023a] Jin, C., Guo, Z., Lin, Y., Luo, L., Chen, H., 2023a.  Label-efficient deep learning in medical image analysis: Challenges and future directions.  arXiv preprint arXiv:2303.12484 .
  * Jin et al. [2019] Jin, K.H., Unser, M., Yi, K.M., 2019.  Self-supervised deep active accelerated mri.  arXiv preprint arXiv:1901.04547 .
  * Jin et al. [2023b] Jin, Q., Li, S., Du, X., Yuan, M., Wang, M., Song, Z., 2023b.  Density-based one-shot active learning for image segmentation.  Engineering Applications of Artificial Intelligence 126, 106805.  doi:10.1016/j.engappai.2023.106805.
  * Jin et al. [2022a] Jin, Q., Yuan, M., Li, S., Wang, H., Wang, M., Song, Z., 2022a.  Cold-start active learning for image classification.  Information Sciences 616, 16–36.  doi:10.1016/j.ins.2022.10.066.
  * Jin et al. [2022b] Jin, Q., Yuan, M., Qiao, Q., Song, Z., 2022b.  One-shot active learning for image segmentation via contrastive learning and diversity-based sampling.  Knowledge-Based Systems 241, 108278\.  doi:10.1016/j.knosys.2022.108278.
  * Jin et al. [2022c] Jin, Q., Yuan, M., Wang, H., Wang, M., Song, Z., 2022c.  Deep active learning models for imbalanced image classification.  Knowledge-Based Systems 257, 109817\.  doi:10.1016/j.knosys.2022.109817.
  * Joshi et al. [2009] Joshi, A.J., Porikli, F., Papanikolopoulos, N., 2009.  Multi-class active learning for image classification, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2372–2379.  doi:10.1109/CVPR.2009.5206627.
  * Jung et al. [2023] Jung, S., Kim, S., Lee, J., 2023.  A simple yet powerful deep active learning with snapshots ensembles, in: International Conference on Learning Representations.
  * Karamcheti et al. [2021] Karamcheti, S., Krishna, R., Fei-Fei, L., Manning, C., 2021\.  Mind your outliers! investigating the negative impact of outliers on active learning for visual question answering, in: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics, Online. pp. 7265–7281.  doi:10.18653/v1/2021.acl-long.564.
  * Kasarla et al. [2019] Kasarla, T., Nagendar, G., Hegde, G.M., Balasubramanian, V., Jawahar, C., 2019.  Region-based active learning for efficient labeling in semantic segmentation, in: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1109–1117.  doi:10.1109/WACV.2019.00123.
  * Kazerouni et al. [2023] Kazerouni, A., Aghdam, E.K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., Merhof, D., 2023.  Diffusion models in medical imaging: A comprehensive survey.  Medical Image Analysis , 102846.
  * Kendall and Gal [2017] Kendall, A., Gal, Y., 2017.  What uncertainties do we need in bayesian deep learning for computer vision?  Advances in neural information processing systems 30\.
  * Kim et al. [2023] Kim, H., Oh, M., Hwang, S., Kwak, S., Ok, J., 2023\.  Adaptive superpixel for active learning in semantic segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 943–953.
  * Kim et al. [2021] Kim, K., Park, D., Kim, K.I., Chun, S.Y., 2021.  Task-aware variational adversarial active learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8166–8175.
  * Kingma and Welling [2013] Kingma, D.P., Welling, M., 2013\.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114 .
  * Kirillov et al. [2023] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al., 2023\.  Segment anything.  arXiv preprint arXiv:2304.02643 .
  * Kirsch et al. [2019] Kirsch, A., van Amersfoort, J., Gal, Y., 2019.  Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.
  * Koh and Liang [2017] Koh, P.W., Liang, P., 2017.  Understanding black-box predictions via influence functions, in: International conference on machine learning, PMLR. pp. 1885–1894.
  * Kohlberger et al. [2012] Kohlberger, T., Singh, V., Alvino, C., Bahlmann, C., Grady, L., 2012.  Evaluating segmentation error without ground truth, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 528–536.
  * Kothawade et al. [2021] Kothawade, S., Beck, N., Killamsetty, K., Iyer, R., 2021\.  Similar: Submodular information measures based active learning in realistic scenarios, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 18685–18697.
  * Kothawade et al. [2022a] Kothawade, S., Ghosh, S., Shekhar, S., Xiang, Y., Iyer, R., 2022a.  Talisman: Targeted active learning for object detection with rare classes and slices using submodular mutual information, in: Computer Vision – ECCV 2022, Springer, Cham. pp. 1–16.  doi:10.1007/978-3-031-19839-7_1.
  * Kothawade et al. [2022b] Kothawade, S., Kaushal, V., Ramakrishnan, G., Bilmes, J., Iyer, R., 2022b.  Prism: A rich class of parameterized submodular information measures for guided data subset selection, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10238–10246.
  * Kothawade et al. [2022c] Kothawade, S., Savarkar, A., Iyer, V., Ramakrishnan, G., Iyer, R., 2022c.  Clinical: Targeted active learning for imbalanced medical image classification, in: Workshop on Medical Image Learning with Limited and Noisy Data, Springer. pp. 119–129.
  * Kovashka et al. [2016] Kovashka, A., Russakovsky, O., Fei-Fei, L., Grauman, K., et al., 2016\.  Crowdsourcing in computer vision.  Foundations and Trends® in computer graphics and Vision 10, 177–243.
  * Kuo et al. [2018] Kuo, W., Häne, C., Yuh, E., Mukherjee, P., Malik, J., 2018.  Cost-sensitive active learning for intracranial hemorrhage detection, in: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham. volume 11072, pp. 715–723.  doi:10.1007/978-3-030-00931-1_82.
  * LeCun et al. [2006] LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., Huang, F., 2006.  A tutorial on energy-based learning.  Predicting structured data 1.
  * Lee et al. [2013] Lee, D.H., et al., 2013.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks, in: Workshop on challenges in representation learning, ICML, Atlanta. p. 896.
  * Lee et al. [2015] Lee, K., Zlateski, A., Ashwin, V., Seung, H.S., 2015\.  Recursive training of 2d-3d convolutional networks for neuronal boundary prediction.  Advances in Neural Information Processing Systems 28\.
  * Lewis and Catlett [1994] Lewis, D.D., Catlett, J., 1994\.  Heterogeneous uncertainty sampling for supervised learning, in: Cohen, W.W., Hirsh, H. (Eds.), Machine Learning Proceedings 1994. Morgan Kaufmann, San Francisco (CA), pp. 148–156.  doi:10.1016/B978-1-55860-335-6.50026-X.
  * Li and Yin [2020] Li, H., Yin, Z., 2020.  Attention, suggestion and annotation: A deep active learning framework for biomedical image segmentation, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Springer International Publishing, Cham. pp. 3–13.  doi:10.1007/978-3-030-59710-8_1.
  * Li et al. [2022] Li, W., Li, J., Wang, Z., Polson, J., Sisk, A.E., Sajed, D.P., Speier, W., Arnold, C.W., 2022.  Pathal: An active learning framework for histopathology image analysis.  IEEE Transactions on Medical Imaging 41, 1176–1187.  doi:10.1109/TMI.2021.3135002.
  * Lin et al. [2020] Lin, Z., Wei, D., Jang, W.D., Zhou, S., Chen, X., Wang, X., Schalek, R., Berger, D., Matejek, B., Kamentsky, L., Peleg, A., Haehn, D., Jones, T., Parag, T., Lichtman, J., Pfister, H., 2020.  Two stream active query suggestion for active learning in connectomics, in: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (Eds.), Computer Vision – ECCV 2020, Springer International Publishing, Cham. pp. 103–120.  doi:10.1007/978-3-030-58523-5_7.
  * Liu et al. [2022a] Liu, G., van Kaick, O., Huang, H., Hu, R., 2022a.  Active self-training for weakly supervised 3d scene semantic segmentation.  arXiv preprint arXiv:2209.07069 .
  * Liu et al. [2020] Liu, J., Cao, L., Tian, Y., 2020.  Deep active learning for effective pulmonary nodule detection, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Springer International Publishing, Cham. pp. 609–618.  doi:10.1007/978-3-030-59725-2_59.
  * Liu et al. [2022b] Liu, P., Wang, L., Ranjan, R., He, G., Zhao, L., 2022b.  A survey on active deep learning: from model driven to data driven.  ACM Computing Surveys (CSUR) 54, 1–34.
  * Liu et al. [2023] Liu, S., Yin, S., Qu, L., Wang, M., Song, Z., 2023.  A structure-aware framework of unsupervised cross-modality domain adaptation via frequency and spatial knowledge distillation.  IEEE Transactions on Medical Imaging .
  * Liu et al. [2021a] Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., Tang, J., 2021a.  Self-supervised learning: Generative or contrastive.  IEEE transactions on knowledge and data engineering 35, 857–876.
  * Liu et al. [2021b] Liu, Z., Ding, H., Zhong, H., Li, W., Dai, J., He, C., 2021b.  Influence selection for active learning, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9274–9283.
  * Liu et al. [2019] Liu, Z., Wang, J., Gong, S., Lu, H., Tao, D., 2019\.  Deep reinforcement active learning for human-in-the-loop person re-identification, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6122–6131.
  * Lou et al. [2023] Lou, W., Li, H., Li, G., Han, X., Wan, X., 2023.  Which pixel to annotate: A label-efficient nuclei segmentation framework.  IEEE Transactions on Medical Imaging 42, 947–958.  doi:10.1109/TMI.2022.3221666.
  * Lyu et al. [2023] Lyu, M., Zhou, J., Chen, H., Huang, Y., Yu, D., Li, Y., Guo, Y., Guo, Y., Xiang, L., Ding, G., 2023\.  Box-level active detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23766–23775.
  * Mackowiak et al. [2018] Mackowiak, R., Lenz, P., Ghori, O., Diego, F., Lange, O., Rother, C., 2018\.  Cereals - cost-effective region-based active learning for semantic segmentation, in: 29th British Machine Vision Conference.
  * Mahapatra et al. [2018] Mahapatra, D., Bozorgtabar, B., Thiran, J.P., Reyes, M., 2018\.  Efficient active learning for image classification and segmentation using a sample selection and conditional generative adversarial network, in: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham. volume 11071, pp. 580–588.  doi:10.1007/978-3-030-00934-2_65.
  * Mahapatra et al. [2022] Mahapatra, D., Poellinger, A., Reyes, M., 2022.  Graph node based interpretability guided sample selection for active learning.  IEEE Transactions on Medical Imaging , 1–1doi:10.1109/TMI.2022.3215017.
  * Mahapatra et al. [2021] Mahapatra, D., Poellinger, A., Shao, L., Reyes, M., 2021\.  Interpretability-driven sample selection using self supervised learning for disease classification and segmentation.  IEEE Transactions on Medical Imaging 40, 2548–2562.  doi:10.1109/TMI.2021.3061724.
  * Mahmood et al. [2022] Mahmood, R., Fidler, S., Law, M.T., 2022.  Low-budget active learning via wasserstein distance: An integer programming approach, in: International Conference on Learning Representations.
  * Mehrtash et al. [2020] Mehrtash, A., Wells, W.M., Tempany, C.M., Abolmaesumi, P., Kapur, T., 2020.  Confidence calibration and predictive uncertainty estimation for deep medical image segmentation.  IEEE transactions on medical imaging 39, 3868–3878.
  * Menze et al. [2014] Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al., 2014\.  The multimodal brain tumor image segmentation benchmark (brats).  IEEE transactions on medical imaging 34, 1993–2024.
  * Mi et al. [2020] Mi, L., Wang, H., Meirovitch, Y., Schalek, R., Turaga, S.C., Lichtman, J.W., Samuel, A.D.T., Shavit, N., 2020\.  Learning guided electron microscopy with active acquisition, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Springer International Publishing, Cham. pp. 77–87.  doi:10.1007/978-3-030-59722-1_8.
  * Miyato et al. [2018] Miyato, T., Maeda, S.i., Koyama, M., Ishii, S., 2018\.  Virtual adversarial training: a regularization method for supervised and semi-supervised learning.  IEEE transactions on pattern analysis and machine intelligence 41, 1979–1993.
  * Moosavi-Dezfooli et al. [2016] Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P., 2016.  Deepfool: a simple and accurate method to fool deep neural networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2574–2582.
  * Munjal et al. [2022] Munjal, P., Hayat, N., Hayat, M., Sourati, J., Khan, S., 2022.  Towards robust and reproducible active learning using neural networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 223–232.
  * Nath et al. [2021] Nath, V., Yang, D., Landman, B.A., Xu, D., Roth, H.R., 2021.  Diminishing uncertainty within the training pool: Active learning for medical image segmentation.  IEEE Transactions on Medical Imaging 40, 2534–2547.  doi:10.1109/TMI.2020.3048055.
  * Nath et al. [2022] Nath, V., Yang, D., Roth, H.R., Xu, D., 2022.  Warm start active learning with proxy labels and selection via semi-supervised fine-tuning, in: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Springer Nature Switzerland, Cham. pp. 297–308.  doi:10.1007/978-3-031-16452-1_29.
  * Nguyen et al. [2021] Nguyen, C., Huynh, M.T., Tran, M.Q., Nguyen, N.H., Jain, M., Ngo, V.D., Vo, T.D., Bui, T., Truong, S.Q.H., 2021\.  Goal: Gist-set online active learning for efficient chest x-ray image annotation, in: Medical Imaging with Deep Learning.
  * Ning et al. [2022] Ning, K.P., Zhao, X., Li, Y., Huang, S.J., 2022.  Active learning for open-set annotation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 41–49.
  * Ning et al. [2021] Ning, M., Lu, D., Wei, D., Bian, C., Yuan, C., Yu, S., Ma, K., Zheng, Y., 2021.  Multi-anchor active domain adaptation for semantic segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9112–9122.
  * Noroozi and Favaro [2016] Noroozi, M., Favaro, P., 2016\.  Unsupervised learning of visual representations by solving jigsaw puzzles, in: European conference on computer vision, Springer. pp. 69–84.
  * OpenAI [2023] OpenAI, 2023.  Gpt-4 technical report.  arXiv:2303.08774.
  * Park et al. [2023] Park, Y., Kim, S., Choi, W., Han, D.J., Moon, J., 2023\.  Active learning for object detection with evidential deep learning and hierarchical uncertainty aggregation, in: International Conference on Learning Representations.
  * Parvaneh et al. [2022] Parvaneh, A., Abbasnejad, E., Teney, D., Haffari, G.R., van den Hengel, A., Shi, J.Q., 2022\.  Active learning by feature mixing, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12237--12246.
  * Payer et al. [2019] Payer, C., Štern, D., Bischof, H., Urschler, M., 2019\.  Integrating spatial configuration into heatmap regression based cnns for landmark localization.  Medical image analysis 54, 207--219.
  * Peng et al. [2021] Peng, F., Wang, C., Liu, J., Yang, Z., 2021.  Active learning for lane detection: A knowledge distillation approach, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15152--15161.
  * Pineda et al. [2020] Pineda, L., Basu, S., Romero, A., Calandra, R., Drozdzal, M., 2020.  Active mr k-space sampling with reinforcement learning, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Springer International Publishing, Cham. pp. 23--33.  doi:10.1007/978-3-030-59713-9_3.
  * Pourahmadi et al. [2021] Pourahmadi, K., Nooralinejad, P., Pirsiavash, H., 2021.  A simple baseline for low-budget active learning.  arXiv preprint arXiv:2110.12033 .
  * Prabhu et al. [2021] Prabhu, V., Chandrasekaran, A., Saenko, K., Hoffman, J., 2021\.  Active domain adaptation via clustering uncertainty-weighted embeddings, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8505--8514.
  * Qi et al. [2019] Qi, Q., Li, Y., Wang, J., Zheng, H., Huang, Y., Ding, X., Rohde, G.K., 2019\.  Label-efficient breast cancer histopathological image classification.  IEEE Journal of Biomedical and Health Informatics 23, 2108--2116.  doi:10.1109/JBHI.2018.2885134.
  * Qin et al. [2018] Qin, C., Schlemper, J., Caballero, J., Price, A.N., Hajnal, J.V., Rueckert, D., 2018\.  Convolutional recurrent neural networks for dynamic mr image reconstruction.  IEEE transactions on medical imaging 38, 280--290.
  * Qu et al. [2022] Qu, L., Liu, S., Liu, X., Wang, M., Song, Z., 2022.  Towards label-efficient automatic diagnosis and analysis: a comprehensive survey of advanced deep learning-based weakly-supervised, semi-supervised and self-supervised techniques in histopathological image analysis.  Physics in Medicine & Biology .
  * Qu et al. [2023] Qu, L., Ma, Y., Yang, Z., Wang, M., Song, Z., 2023.  Openal: An efficient deep active learning framework for open-set pathology image classification, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 3--13.
  * Quan et al. [2022] Quan, Q., Yao, Q., Li, J., Zhou, S.K., 2022.  Which images to label for few-shot medical landmark detection?, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20606--20616.
  * Radford et al. [2021] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al., 2021\.  Learning transferable visual models from natural language supervision, in: International conference on machine learning, PMLR. pp. 8748--8763.
  * Rädsch et al. [2023] Rädsch, T., Reinke, A., Weru, V., Tizabi, M.D., Schreck, N., Kavur, A.E., Pekdemir, B., Roß, T., Kopp-Schneider, A., Maier-Hein, L., 2023\.  Labelling instructions matter in biomedical image analysis.  Nature Machine Intelligence 5, 273--283.
  * Rajpurkar et al. [2022] Rajpurkar, P., Chen, E., Banerjee, O., Topol, E.J., 2022\.  Ai in health and medicine.  Nature medicine 28, 31--38.
  * Rangwani et al. [2021] Rangwani, H., Jain, A., Aithal, S.K., Babu, R.V., 2021\.  S3vaada: Submodular subset selection for virtual adversarial active domain adaptation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7516--7525.
  * Ren et al. [2021] Ren, P., Xiao, Y., Chang, X., Huang, P.Y., Li, Z., Gupta, B.B., Chen, X., Wang, X., 2021.  A survey of deep active learning.  ACM computing surveys (CSUR) 54, 1--40.
  * Rombach et al. [2022] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B., 2022.  High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684--10695.
  * Roth and Small [2006] Roth, D., Small, K., 2006.  Margin-based active learning for structured output spaces, in: Hutchison, D., Kanade, T., Kittler, J., Kleinberg, J.M., Mattern, F., Mitchell, J.C., Naor, M., Nierstrasz, O., Pandu Rangan, C., Steffen, B., Sudan, M., Terzopoulos, D., Tygar, D., Vardi, M.Y., Weikum, G., Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (Eds.), Machine Learning: ECML 2006. Springer Berlin Heidelberg, Berlin, Heidelberg. volume 4212, pp. 413--424.  doi:10.1007/11871842_40.
  * Sadafi et al. [2019] Sadafi, A., Koehler, N., Makhro, A., Bogdanova, A., Navab, N., Marr, C., Peng, T., 2019.  Multiclass deep active learning for detecting red blood cell subtypes in brightfield microscopy, in: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.T., Khan, A. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, Springer International Publishing, Cham. pp. 685--693.  doi:10.1007/978-3-030-32239-7_76.
  * Sadafi et al. [2023] Sadafi, A., Navab, N., Marr, C., 2023.  Active learning enhances classification of histopathology whole slide images with attention-based multiple instance learning, in: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pp. 1--5.  doi:10.1109/ISBI53787.2023.10230685.
  * Saquil et al. [2018] Saquil, Y., Kim, K.I., Hall, P., 2018.  Ranking cgans: Subjective control over semantic image attributes.  arXiv preprint arXiv:1804.04082 .
  * Sener and Savarese [2018] Sener, O., Savarese, S., 2018\.  Active learning for convolutional neural networks: A core-set approach, in: International Conference on Learning Representations.
  * Sensoy et al. [2018] Sensoy, M., Kaplan, L., Kandemir, M., 2018.  Evidential deep learning to quantify classification uncertainty.  Advances in neural information processing systems 31\.
  * Settles [2009] Settles, B., 2009.  Active learning literature survey .
  * Seung et al. [1992] Seung, H.S., Opper, M., Sompolinsky, H., 1992.  Query by committee, in: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, Association for Computing Machinery, New York, NY, USA. pp. 287--294.  doi:10.1145/130385.130417.
  * Shaham et al. [2019] Shaham, T.R., Dekel, T., Michaeli, T., 2019.  Singan: Learning a generative model from a single natural image, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 4570--4580.
  * Shen et al. [2020] Shen, H., Tian, K., Dong, P., Zhang, J., Yan, K., Che, S., Yao, J., Luo, P., Han, X., 2020.  Deep active learning for breast cancer segmentation on immunohistochemistry images, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Springer International Publishing, Cham. pp. 509--518.  doi:10.1007/978-3-030-59722-1_49.
  * Shin et al. [2021] Shin, I., Kim, D.J., Cho, J.W., Woo, S., Park, K., Kweon, I.S., 2021.  Labor: Labeling only if required for domain adaptive semantic segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8588--8598.
  * Shui et al. [2020] Shui, C., Zhou, F., Gagné, C., Wang, B., 2020\.  Deep active learning: Unified and principled method for query and training, in: Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR. pp. 1308--1318.
  * Siddiqui et al. [2020] Siddiqui, Y., Valentin, J., Niessner, M., 2020.  Viewal: Active learning with viewpoint entropy for semantic segmentation, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Seattle, WA, USA. pp. 9430--9440.  doi:10.1109/CVPR42600.2020.00945.
  * Sim et al. [2020] Sim, Y., Chung, M.J., Kotter, E., Yune, S., Kim, M., Do, S., Han, K., Kim, H., Yang, S., Lee, D.J., et al., 2020.  Deep convolutional neural network--based software improves radiologist detection of malignant lung nodules on chest radiographs.  Radiology 294, 199--209.
  * Sinha et al. [2019] Sinha, S., Ebrahimi, S., Darrell, T., 2019.  Variational adversarial active learning, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5972--5981.
  * Sourati et al. [2017] Sourati, J., Akcakaya, M., Leen, T.K., Erdogmus, D., Dy, J.G., 2017.  Asymptotic analysis of objectives based on fisher information in active learning.  The Journal of Machine Learning Research 18, 1123--1163.
  * Sourati et al. [2018] Sourati, J., Gholipour, A., Dy, J.G., Kurugol, S., Warfield, S.K., 2018.  Active deep learning with fisher information for patch-wise semantic segmentation, in: Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R., Bradley, A., Papa, J.P., Belagiannis, V., Nascimento, J.C., Lu, Z., Conjeti, S., Moradi, M., Greenspan, H., Madabhushi, A. (Eds.), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer International Publishing, Cham. volume 11045, pp. 83--91.  doi:10.1007/978-3-030-00889-5_10.
  * Sourati et al. [2019] Sourati, J., Gholipour, A., Dy, J.G., Tomas-Fernandez, X., Kurugol, S., Warfield, S.K., 2019\.  Intelligent labeling based on fisher information for medical image segmentation using deep learning.  IEEE Transactions on Medical Imaging 38, 2642--2653.  doi:10.1109/TMI.2019.2907805.
  * Su et al. [2020] Su, J.C., Tsai, Y.H., Sohn, K., Liu, B., Maji, S., Chandraker, M., 2020.  Active adversarial domain adaptation, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 739--748.
  * Sun et al. [2023] Sun, S., Zhi, S., Heikkilä, J., Liu, L., 2023\.  Evidential uncertainty and diversity guided active learning for scene graph generation, in: The Eleventh International Conference on Learning Representations.
  * Tajbakhsh et al. [2020] Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J.N., Wu, Z., Ding, X., 2020.  Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation.  Medical Image Analysis 63, 101693\.
  * Takezoe et al. [2023] Takezoe, R., Liu, X., Mao, S., Chen, M.T., Feng, Z., Zhang, S., Wang, X., et al., 2023\.  Deep active learning for computer vision: Past and future.  APSIPA Transactions on Signal and Information Processing 12.
  * Tang et al. [2022] Tang, C., Xie, L., Zhang, G., Zhang, X., Tian, Q., Hu, X., 2022.  Active pointly-supervised instance segmentation, in: European Conference on Computer Vision, Springer. pp. 606--623.
  * Tarvainen and Valpola [2017] Tarvainen, A., Valpola, H., 2017\.  Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results.  Advances in neural information processing systems 30\.
  * Tolkach et al. [2020] Tolkach, Y., Dohmgörgen, T., Toma, M., Kristiansen, G., 2020\.  High-accuracy prostate cancer pathology using deep learning.  Nature Machine Intelligence 2, 411--418.
  * Tran et al. [2019] Tran, T., Do, T.T., Reid, I., Carneiro, G., 2019.  Bayesian generative active deep learning, in: Proceedings of the 36th International Conference on Machine Learning, PMLR. pp. 6295--6304.
  * Tschandl et al. [2020] Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al., 2020\.  Human--computer collaboration for skin cancer recognition.  Nature Medicine 26, 1229--1234.
  * Unnikrishnan et al. [2021] Unnikrishnan, B., Nguyen, C., Balaram, S., Li, C., Foo, C.S., Krishnaswamy, P., 2021\.  Semi-supervised classification of radiology images with noteacher: A teacher that is not mean.  Medical Image Analysis 73, 102148\.
  * Vo et al. [2022] Vo, H.V., Siméoni, O., Gidaris, S., Bursuc, A., Pérez, P., Ponce, J., 2022\.  Active learning strategies for weakly-supervised object detection, in: European Conference on Computer Vision, Springer. pp. 211--230.
  * Wan et al. [2023] Wan, F., Ye, Q., Yuan, T., Xu, S., Liu, J., Ji, X., Huang, Q., 2023.  Multiple instance differentiation learning for active object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence , 1--15doi:10.1109/TPAMI.2023.3277738.
  * Wang et al. [2022a] Wang, C., Shang, K., Zhang, H., Zhao, S., Liang, D., Zhou, S.K., 2022a.  Active ct reconstruction with a learned sampling policy.  arXiv preprint arXiv:2211.01670 .
  * Wang et al. [2020a] Wang, J., Yan, Y., Zhang, Y., Cao, G., Yang, M., Ng, M.K., 2020a.  Deep reinforcement active learning for medical image classification, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Springer International Publishing, Cham. pp. 33--42.  doi:10.1007/978-3-030-59710-8_4.
  * Wang et al. [2017] Wang, K., Zhang, D., Li, Y., Zhang, R., Lin, L., 2017\.  Cost-effective active learning for deep image classification.  IEEE Transactions on Circuits and Systems for Video Technology 27, 2591--2600.  doi:10.1109/TCSVT.2016.2589879.
  * Wang et al. [2018] Wang, P., Xiao, X., Glissen Brown, J.R., Berzin, T.M., Tu, M., Xiong, F., Hu, X., Liu, P., Song, Y., Zhang, D., et al., 2018.  Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy.  Nature biomedical engineering 2, 741--748.
  * Wang et al. [2020b] Wang, S., Li, Y., Ma, K., Ma, R., Guan, H., Zheng, Y., 2020b.  Dual adversarial network for deep active learning, in: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (Eds.), Computer Vision – ECCV 2020, Springer International Publishing, Cham. pp. 680--696.  doi:10.1007/978-3-030-58586-0_40.
  * Wang et al. [2020c] Wang, S., Tarroni, G., Qin, C., Mo, Y., Dai, C., Chen, C., Glocker, B., Guo, Y., Rueckert, D., Bai, W., 2020c.  Deep generative model-based quality control for cardiac mri segmentation, in: Medical Image Computing and Computer Assisted Intervention--MICCAI 2020: 23rd International Conference, Lima, Peru, October 4--8, 2020, Proceedings, Part IV 23, Springer. pp. 88--97.
  * Wang et al. [2022b] Wang, T., Li, X., Yang, P., Hu, G., Zeng, X., Huang, S., Xu, C.Z., Xu, M., 2022b.  Boosting active learning via improving test performance.  Proceedings of the AAAI Conference on Artificial Intelligence 36, 8566--8574.  doi:10.1609/aaai.v36i8.20834.
  * Wang et al. [2022c] Wang, X., Lian, L., Yu, S.X., 2022c.  Unsupervised selective labeling for more effective semi-supervised learning, in: European Conference on Computer Vision, Springer. pp. 427--445.
  * Wang et al. [2021] Wang, Y., Huang, G., Song, S., Pan, X., Xia, Y., Wu, C., 2021.  Regularizing deep networks with semantic data augmentation.  IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 3733--3748.
  * Wang and Yin [2021] Wang, Z., Yin, Z., 2021.  Annotation-efficient cell counting, in: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021, Springer International Publishing, Cham. pp. 405--414.  doi:10.1007/978-3-030-87237-3_39.
  * Wei et al. [2015] Wei, K., Iyer, R., Bilmes, J., 2015.  Submodularity in data subset selection and active learning, in: Proceedings of the 32nd International Conference on Machine Learning, PMLR. pp. 1954--1963.
  * Williams [1992] Williams, R.J., 1992.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  Machine learning 8, 229--256.
  * Wu et al. [2022a] Wu, J., Chen, J., Huang, D., 2022a.  Entropy-based active learning for object detection with progressive diversity constraint, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9397--9406.
  * Wu et al. [2022b] Wu, T.H., Liou, Y.S., Yuan, S.J., Lee, H.Y., Chen, T.I., Huang, K.C., Hsu, W.H., 2022b.  D2ada: Dynamic density-aware active domain adaptation for semantic segmentation, in: European Conference on Computer Vision, Springer. pp. 449--467.
  * Wu et al. [2021a] Wu, T.H., Liu, Y.C., Huang, Y.K., Lee, H.Y., Su, H.T., Huang, P.C., Hsu, W.H., 2021a.  Redal: Region-based and diversity-aware active learning for point cloud semantic segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15510--15519.
  * Wu et al. [2021b] Wu, X., Chen, C., Zhong, M., Wang, J., Shi, J., 2021b.  Covid-al: The diagnosis of covid-19 with deep active learning.  Medical Image Analysis 68, 101913\.  doi:10.1016/j.media.2020.101913.
  * Wu et al. [2022c] Wu, Y., Zheng, B., Chen, J., Chen, D.Z., Wu, J., 2022c.  Self-learning and one-shot learning based single-slice annotation for 3d medical image segmentation, in: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Springer Nature Switzerland, Cham. pp. 244--254.  doi:10.1007/978-3-031-16452-1_24.
  * Xie et al. [2022a] Xie, B., Yuan, L., Li, S., Liu, C.H., Cheng, X., 2022a.  Towards fewer annotations: Active learning via region impurity and prediction uncertainty for domain adaptive semantic segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8068--8078.
  * Xie et al. [2022b] Xie, B., Yuan, L., Li, S., Liu, C.H., Cheng, X., Wang, G., 2022b.  Active learning for domain adaptation: An energy-based approach, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8708--8716.  doi:10.1609/aaai.v36i8.20850.
  * Xie et al. [2022c] Xie, M., Li, S., Zhang, R., Liu, C.H., 2022c.  Dirichlet-based uncertainty calibration for active domain adaptation, in: The Eleventh International Conference on Learning Representations.
  * Xie et al. [2022d] Xie, M., Li, Y., Wang, Y., Luo, Z., Gan, Z., Sun, Z., Chi, M., Wang, C., Wang, P., 2022d.  Learning distinctive margin toward active domain adaptation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7993--8002.
  * Xie et al. [2023a] Xie, Y., Ding, M., Tomizuka, M., Zhan, W., 2023a.  Towards free data selection with general-purpose models, in: Thirty-seventh Conference on Neural Information Processing Systems.  URL: https://openreview.net/forum?id=KBXcDAaZE7.
  * Xie et al. [2023b] Xie, Y., Lu, H., Yan, J., Yang, X., Tomizuka, M., Zhan, W., 2023b.  Active finetuning: Exploiting annotation budget in the pretraining-finetuning paradigm, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23715--23724.
  * Xu et al. [2018] Xu, X., Lu, Q., Yang, L., Hu, S., Chen, D., Hu, Y., Shi, Y., 2018.  Quantization of fully convolutional networks for accurate biomedical image segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8300--8308.
  * Xu et al. [2021] Xu, Y., Xu, X., Jin, L., Gao, S., Goh, R.S.M., Ting, D.S.W., Liu, Y., 2021\.  Partially-supervised learning for vessel segmentation in ocular images, in: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021, Springer International Publishing, Cham. pp. 271--281.  doi:10.1007/978-3-030-87193-2_26.
  * Yang et al. [2017] Yang, L., Zhang, Y., Chen, J., Zhang, S., Chen, D.Z., 2017\.  Suggestive annotation: A deep active learning framework for biomedical image segmentation, in: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (Eds.), Medical Image Computing and Computer Assisted Intervention - MICCAI 2017, Springer International Publishing, Cham. pp. 399--407.  doi:10.1007/978-3-319-66179-7_46.
  * Yehuda et al. [2022] Yehuda, O., Dekel, A., Hacohen, G., Weinshall, D., 2022\.  Active learning through a covering lens, in: Advances in Neural Information Processing Systems.
  * Yi et al. [2022] Yi, J.S.K., Seo, M., Park, J., Choi, D.G., 2022.  Pt4al: Using self-supervised pretext tasks for active learning, in: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (Eds.), Computer Vision – ECCV 2022, Springer Nature Switzerland, Cham. pp. 596--612.  doi:10.1007/978-3-031-19809-0_34.
  * Yoo and Kweon [2019] Yoo, D., Kweon, I.S., 2019.  Learning loss for active learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 93--102.
  * Yuan et al. [2023] Yuan, J., Zhang, B., Yan, X., Chen, T., Shi, B., Li, Y., Qiao, Y., 2023.  Bi3d: Bi-domain active learning for cross-domain 3d object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15599--15608.
  * Yuan et al. [2020] Yuan, M., Lin, H.T., Boyd-Graber, J., 2020.  Cold-start active learning through self-supervised language modeling, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online. pp. 7935--7948.  doi:10.18653/v1/2020.emnlp-main.637.
  * Yuan et al. [2021] Yuan, T., Wan, F., Fu, M., Liu, J., Xu, S., Ji, X., Ye, Q., 2021.  Multiple instance active learning for object detection, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5330--5339.
  * Zhan et al. [2022] Zhan, X., Wang, Q., Huang, K.h., Xiong, H., Dou, D., Chan, A.B., 2022.  A comparative survey of deep active learning.  arXiv preprint arXiv:2203.13450 .
  * Zhang et al. [2020] Zhang, B., Li, L., Yang, S., Wang, S., Zha, Z.J., Huang, Q., 2020.  State-relabeling adversarial active learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8756--8765.
  * Zhang et al. [2021] Zhang, B., Wang, Y., Hou, W., Wu, H., Wang, J., Okumura, M., Shinozaki, T., 2021\.  Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling.  Advances in Neural Information Processing Systems 34, 18408--18419.
  * Zhang et al. [2016] Zhang, R., Isola, P., Efros, A.A., 2016.  Colorful image colorization, in: Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, Springer. pp. 649--666.
  * Zhang et al. [2022a] Zhang, W., Zhu, L., Hallinan, J., Zhang, S., Makmur, A., Cai, Q., Ooi, B.C., 2022a.  Boostmis: Boosting medical image semi-supervised learning with adaptive pseudo labeling and informative active annotation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20666--20676.
  * Zhang et al. [2023] Zhang, Y., Kang, B., Hooi, B., Yan, S., Feng, J., 2023\.  Deep long-tailed learning: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence .
  * Zhang et al. [2022b] Zhang, Y., Zhang, X., Xie, L., Li, J., Qiu, R.C., Hu, H., Tian, Q., 2022b.  One-bit active query with contrastive pairs, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9697--9705.
  * Zhang et al. [2019] Zhang, Z., Romero, A., Muckley, M.J., Vincent, P., Yang, L., Drozdzal, M., 2019\.  Reducing uncertainty in undersampled mri reconstruction with active acquisition, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Long Beach, CA, USA. pp. 2049--2053.  doi:10.1109/CVPR.2019.00215.
  * Zhao et al. [2021] Zhao, Z., Zeng, Z., Xu, K., Chen, C., Guan, C., 2021\.  Dsal: Deeply supervised active learning from strong and weak labelers for biomedical image segmentation.  IEEE Journal of Biomedical and Health Informatics 25, 3744--3751.  doi:10.1109/JBHI.2021.3052320.
  * Zheng et al. [2019] Zheng, H., Yang, L., Chen, J., Han, J., Zhang, Y., Liang, P., Zhao, Z., Wang, C., Chen, D.Z., 2019\.  Biomedical image segmentation via representative annotation.  Proceedings of the AAAI Conference on Artificial Intelligence 33, 5901--5908.  doi:10.1609/aaai.v33i01.33015901.
  * Zheng et al. [2020] Zheng, H., Zhang, Y., Yang, L., Wang, C., Chen, D.Z., 2020.  An annotation sparsification strategy for 3d medical image segmentation via representative selection and self-training.  Proceedings of the AAAI Conference on Artificial Intelligence 34, 6925--6932.  doi:10.1609/aaai.v34i04.6175.
  * Zhou et al. [2016] Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A., 2016.  Learning deep features for discriminative localization, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921--2929.
  * Zhou et al. [2021a] Zhou, T., Li, L., Bredell, G., Li, J., Konukoglu, E., 2021a.  Quality-aware memory network for interactive volumetric image segmentation, in: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021, Springer International Publishing, Cham. pp. 560--570.  doi:10.1007/978-3-030-87196-3_52.
  * Zhou et al. [2022] Zhou, T., Li, L., Bredell, G., Li, J., Konukoglu, E., 2022\.  Volumetric memory network for interactive medical image segmentation.  Medical Image Analysis , 102599doi:10.1016/j.media.2022.102599.
  * Zhou et al. [2017] Zhou, Z., Shin, J., Zhang, L., Gurudu, S., Gotway, M., Liang, J., 2017.  Fine-tuning convolutional neural networks for biomedical image analysis: Actively and incrementally, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7340--7351.
  * Zhou et al. [2021b] Zhou, Z., Shin, J.Y., Gurudu, S.R., Gotway, M.B., Liang, J., 2021b.  Active, continual fine tuning of convolutional neural networks for reducing annotation efforts.  Medical Image Analysis 71, 101997\.  doi:10.1016/j.media.2021.101997.
  * Zhu and Bento [2017] Zhu, J.J., Bento, J., 2017.  Generative adversarial active learning.  arXiv preprint arXiv:1702.07956 .