* [117] Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al.  Concept embedding models.  arXiv preprint arXiv:2209.09056, 2022.
  * [118] Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin.  From attribution maps to human-understandable explanations through Concept Relevance Propagation.  Nature Machine Intelligence, 5(9):1006–1019, September 2023.  Number: 9, Publisher: Nature Publishing Group.
  * [119] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu.  The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [120] Md Kamruzzaman Sarker, Lu Zhou, Aaron Eberhart, and Pascal Hitzler.  Neuro-symbolic artificial intelligence.  AI Communications, 34(3):197–209, 2021.
  * [121] Kyle Hamilton, Aparna Nayak, Bojan Božić, and Luca Longo.  Is neuro-symbolic ai meeting its promises in natural language processing? a structured review.  Semantic Web, 15(Preprint):1–42, 2022.
  * [122] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick.  Clevr: A diagnostic dataset for compositional language and elementary visual reasoning.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2901–2910, 2017.
  * [123] Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum.  CLEVRER: collision events for video representation and reasoning.  In ICLR, 2020.
  * [124] Heimo Müller and Andreas Holzinger.  Kandinsky patterns.  Artificial intelligence, 300:103546, 2021.
  * [125] Harm de Vries, Dzmitry Bahdanau, Shikhar Murty, Aaron C. Courville, and Philippe Beaudoin.  CLOSURE: assessing systematic generalization of CLEVR models.  In Visually Grounded Interaction and Language (ViGIL), NeurIPS 2019 Workshop, Vancouver, Canada, December 13, 2019, 2019.
  * [126] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman.  Deep inside convolutional networks: Visualising image classification models and saliency maps.  In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Workshop Track Proceedings, 2014.
  * [127] Marco Ribeiro, Sameer Singh, and Carlos Guestrin.  “why should I trust you?”: Explaining the predictions of any classifier.  In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 97–101, San Diego, California, June 2016. Association for Computational Linguistics.
  * [128] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In European conference on computer vision, pages 818–833. Springer, 2014.
  * [129] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In IEEE International Conference on Computer Vision (ICCV), pages 618–626, 2017.
  * [130] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  PloS one, 10(7), 2015.
  * [131] Johannes Schneider and Giovanni Apruzzese.  Concept-based adversarial attacks: Tricking humans and classifiers alike.  In 2022 IEEE Security and Privacy Workshops (SPW), pages 66–72. IEEE, 2022.
  * [132] Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune.  Synthesizing the preferred inputs for neurons in neural networks via deep generator networks.  In Advances in neural information processing systems, pages 3387–3395, 2016.
  * [133] Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell.  Toward transparent ai: A survey on interpreting the inner structures of deep neural networks.  In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 464–483. IEEE, 2023.
  * [134] Johannes Schneider and Michalis Vlachos.  A Survey of Deep Learning: From Activations to Transformers.  arXiv preprint arXiv:2302.00722, 2023.
  * [135] Johannes Schneider and Michalis Vlachos.  Explaining classifiers by constructing familiar concepts.  Machine Learning, pages 1–34, 2022.
  * [136] Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Suggala, David I Inouye, and Pradeep K Ravikumar.  On the (in) fidelity and sensitivity of explanations.  Advances in Neural Information Processing Systems, 32, 2019.
  * [137] Leander Weber, Sebastian Lapuschkin, Alexander Binder, and Wojciech Samek.  Beyond explaining: Opportunities and challenges of xai-based model improvement.  Information Fusion, 92:154–176, 2023.
  * [138] Yuyang Gao, Siyi Gu, Junji Jiang, Sungsoo Ray Hong, Dazhou Yu, and Liang Zhao.  Going beyond xai: A systematic survey for explanation-guided learning.  arXiv preprint arXiv:2212.03954, 2022.
  * [139] Andrea Ferrario and Michele Loi.  The robustness of counterfactual explanations over time.  IEEE Access, 10:82736–82750, 2022.
  * [140] Luyu Qiu, Yi Yang, Caleb Chen Cao, Yueyuan Zheng, Hilary Ngai, Janet Hsiao, and Lei Chen.  Generating perturbation-based explanations with robustness to out-of-distribution data.  In Proceedings of the ACM Web Conference 2022, pages 3594–3605, 2022.
  * [141] Dominik Seuß.  Bridging the gap between explainable ai and uncertainty quantification to enhance trustability.  arXiv preprint arXiv:2105.11828, 2021.
  * [142] Aditya Kuppa and Nhien-An Le-Khac.  Black box attacks on explainable artificial intelligence (xai) methods in cyber security.  In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2020.
  * [143] Abdullah Caglar Oksuz, Anisa Halimi, and Erman Ayday.  Autolycus: Exploiting explainable ai (xai) for model extraction attacks against decision tree models.  arXiv preprint arXiv:2302.02162, 2023.
  * [144] Frederik Pahde, Maximilian Dreyer, Wojciech Samek, and Sebastian Lapuschkin.  Reveal to revise: An explainable ai life cycle for iterative bias correction of deep models.  In Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, and Russell Taylor, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, pages 596–606, Cham, 2023. Springer Nature Switzerland.
  * [145] Francesco Bodria, Fosca Giannotti, Riccardo Guidotti, Francesca Naretto, Dino Pedreschi, and Salvatore Rinzivillo.  Benchmarking and survey of explanation methods for black box models.  Data Mining and Knowledge Discovery, 37:1719–1778, 2023.
  * [146] Riccardo Guidotti.  Evaluating local explanation methods on ground truth.  Artif. Intell., 291:103428, 2021.
  * [147] Riccardo Guidotti.  Counterfactual explanations and how to find them: literature review and benchmarking.  Data Mining and Knowledge Discovery, pages 1–55, 2022.
  * [148] Giulia Vilone and Luca Longo.  Notions of explainability and evaluation approaches for explainable artificial intelligence.  Information Fusion, 76:89–106, 2021.
  * [149] Mark T Keane, Eoin M Kenny, Eoin Delaney, and Barry Smyth.  If only we had better counterfactual explanations: Five key deficits to rectify in the evaluation of counterfactual xai techniques.  Proceedings of the 30th International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4466–4474, 2021.
  * [150] Jonathan Dodge, Q. Vera Liao, Yunfeng Zhang, Rachel K. E. Bellamy, and Casey Dugan.  Explaining models: an empirical study of how explanations impact fairness judgment.  In IUI, pages 275–285. ACM, 2019.
  * [151] Ana Lucic, Hinda Haned, and Maarten de Rijke.  Why does my model fail?: contrastive local explanations for retail forecasting.  In FAT*, pages 90–98. ACM, 2020.
  * [152] Carlo Metta, Riccardo Guidotti, Yuan Yin, Patrick Gallinari, and Salvatore Rinzivillo.  Exemplars and counterexemplars explanations for skin lesion classifiers.  In HHAI, volume 354 of Frontiers in Artificial Intelligence and Applications, pages 258–260. IOS Press, 2022.
  * [153] Tim Miller.  Explanation in artificial intelligence: Insights from the social sciences.  Artificial Intelligence, 267:1–38, 2019.
  * [154] Robert R. Hoffman, Shane T. Mueller, Gary Klein, and Jordan Litman.  Metrics for explainable AI: Challenges and prospects.  ArXiv, abs/1812.04608, 2018.
  * [155] Chris van der Lee, Albert Gatt, Emiel van Miltenburg, and Emiel Krahmer.  Human evaluation of automatically generated text: Current trends and best practice guidelines.  Computer Speech & Language, 67:101151, 2021.
  * [156] Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi.  A survey of methods for explaining black box models.  ACM Comput. Surv., 51(5):93:1–93:42, 2019.
  * [157] Alejandro Barredo Arrieta, Natalia Díaz Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera.  Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI.  Inf. Fusion, 58:82–115, 2020.
  * [158] Meike Nauta, Jan Trienes, Shreyasi Pathak, Elisa Nguyen, Michelle Peters, Yasmin Schmitt, Jörg Schlötterer, Maurice van Keulen, and Christin Seifert.  From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai.  ACM Comput. Surv., feb 2023.
  * [159] Anna Hedström, Leander Weber, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina M. C. Höhne.  Quantus: An explainable ai toolkit for responsible evaluation of neural network explanation.  Journal of Machine Learning Research, 24(34):1–11, 2023.
  * [160] Leila Arras, Ahmed Osman, and Wojciech Samek.  Clevr-xai: A benchmark dataset for the ground truth evaluation of neural network explanations.  Information Fusion, 81:14–40, 2022.
  * [161] Frederik Pahde, Maximilian Dreyer, Wojciech Samek, and Sebastian Lapuschkin.  Reveal to revise: An explainable ai life cycle for iterative bias correction of deep models.  In Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, and Russell Taylor, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, pages 596–606, Cham, 2023. Springer Nature Switzerland.
  * [162] Luca Longo.  Formalising human mental workload as non-monotonic concept for adaptive and personalised web-design.  In User Modeling, Adaptation, and Personalization: 20th International Conference, UMAP 2012, Montreal, Canada, July 16-20, 2012. Proceedings 20, pages 369–373. Springer, 2012.
  * [163] Luca Longo.  Designing medical interactive systems via assessment of human mental workload.  In 2015 IEEE 28th International Symposium on Computer-Based Medical Systems, pages 364–365. IEEE, 2015.
  * [164] GM Hancock, L Longo, MS Young, and PA Hancock.  Mental workload.  Handbook of human factors and ergonomics, pages 203–226, 2021.
  * [165] Luca Longo, Christoper D. Wickens, Peter A. Hancock, and Gabriela M. Hancock.  Human mental workload: A survey and a novel inclusive definition.  Frontiers in Psychology, 13, 2022.
  * [166] Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-Lopez, Daniel Molina, Benjamins Richard, Raja Chatila, and Francisco Herrera.  Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI.  Information Fusion, 58:82–115, 2020.
  * [167] Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilović, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, and Yunfeng Zhang.  One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques, 2021.
  * [168] Maya Krishnan.  Against interpretability: A critical examination of the interpretability problem in machine learning.  Philosophy & Technology, 33(3):487–502, 2020.
  * [169] Zachary C Lipton.  The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.  Queue, 16(3):31–57, 2018.
  * [170] Upol Ehsan and Mark O Riedl.  Social construction of xai: Do we need one definition to rule them all?  In Michael Muller, Plamen Angelov, Hal Daume III, Shion Guha, Q. Vera Liao, Nuria Oliver, and David Piorkowski, editors, Proceedings of the NeurIPS 2022 Workshop on Human-Centered AI, 2022.
  * [171] Miruna-Adriana Clinciu and Helen Hastie.  A survey of explainable ai terminology.  In Jose M. Alonso and Alejandro Catala, editors, Proceedings of the 1st Workshop on Interactive Natural Language Technology for Explainable Artificial Intelligence, NL4XAI 2019, pages 8–13, Stroudsburg, PA, USA, 2019. Association for Computational Linguistics.
  * [172] Mara Graziani, Lidia Dutkiewicz, Davide Calvaresi, José Pereira Amorim, Katerina Yordanova, Mor Vered, Rahul Nair, Pedro Henriques Abreu, Tobias Blanke, Valeria Pulignano, John O. Prior, Lode Lauwaert, Wessel Reijers, Adrien Depeursinge, Vincent Andrearczyk, and Henning Müller.  A global taxonomy of interpretable ai: Unifying the terminology for the technical and social sciences.  Artificial Intelligence Review, pages 1–32, 2022.
  * [173] Tathagata Chakraborti, Anagha Kulkarni, Sarath Sreedharan, David E. Smith, and Subbarao Kambhampati.  Explicability? legibility? predictability? transparency? privacy? security? the emerging landscape of interpretable agent behavior.  In J. Benton, Nir Lipovetzky, Eva Onaindia, David E. Smith, and Siddharth Srivastava, editors, Proceedings of the 29th International Conference on Automated Planning and Scheduling, ICAPS 2019, pages 86–96, Palo Alto, CA, USA, 2019. AAAI Press.
  * [174] Sarah Sterz, Kevin Baum, Anne Lauber-Rönsberg, and Holger Hermanns.  Towards perspicuity requirements.  In Tao Yue and Mehdi Mirakhorli, editors, 29th IEEE International Requirements Engineering Conference Workshops, REW 2021, pages 159–163, Piscataway, NJ, USA, 2021. IEEE.
  * [175] AI High-Level Expert Group.  Ethics guidelines for trustworthy AI.  B-1049 Brussels, 2019.
  * [176] Lena Kästner, Markus Langer, Veronika Lazar, Astrid Schomäcker, Timo Speith, and Sarah Sterz.  On the relation of trust and explainability: Why to engineer for trustworthiness.  In Tao Yue and Mehdi Mirakhorli, editors, 29th IEEE International Requirements Engineering Conference Workshops, REW 2021, pages 169–175, Piscataway, NJ, USA, 2021. IEEE.
  * [177] Scott Robbins.  A misdirected principle with a catch: Explicability for ai.  Minds and Machines, 29(4):495–514, 2019.
  * [178] René F. Kizilcec.  How much information? effects of transparency on trust in an algorithmic interface.  In Jofish Kaye, Allison Druin, Cliff Lampe, Dan Morris, and Juan P. Hourcade, editors, Proceedings of the 34th Conference on Human Factors in Computing Systems, CHI 2016, pages 2390–2395, New York, NY, USA, 2016. Association for Computing Machinery.
  * [179] Bishwamittra Ghosh, Dmitry Malioutov, and Kuldeep S. Meel.  Interpretable classification rules in relaxed logical form.  In Tim Miller, Rosina Weber, and Daniele Magazzeni, editors, Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence, IJCAI XAI 2019, pages 14–20, 2019.
  * [180] Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin.  "Why should I trust you?": Explaining the predictions of any classifier.  In Charu Aggarwal, Balaji Krishnapuram, Rajeev Rastogi, Dou Shen, Mohak Shah, and Alex Smola, editors, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pages 1135–1144, New York, NY, USA, 2016. Association for Computing Machinery.
  * [181] Natalia Díaz-Rodríguez, Javier Del Ser, Mark Coeckelbergh, Marcos López de Prado, Enrique Herrera-Viedma, and Francisco Herrera.  Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation.  Inf. Fusion, 99:101896, 2023.
  * [182] Nicola Palladino.  A ‘biased’ emerging governance regime for artificial intelligence? how ai ethics get skewed moving from principles to practices.  Telecommunications Policy, 47(5):102479, 2023.
  * [183] Kareem Khalifa.  Inaugurating understanding or repackaging explanation?  Philosophy of Science, 79(1):15–37, 2012.
  * [184] Michael Strevens.  No understanding without explanation.  Studies in history and philosophy of science Part A, 44(3):510–515, 2013.
  * [185] Peter Lipton.  Understanding without explanation.  Scientific understanding: Philosophical perspectives, pages 43–63, 2009.
  * [186] Catherine Z Elgin.  True enough.  MIT press, 2017.
  * [187] Jonathan Kvanvig.  Responses to critics.  Epistemic value, pages 339–351, 2009.
  * [188] Moti Mizrahi.  Idealizations and scientific understanding.  Philosophical Studies, 160:237–252, 2012.
  * [189] J Adam Carter and Emma C Gordon.  Objectual understanding, factivity and belief.  Epistemic reasons, norms and goals, 423, 2016.
  * [190] Adrian Erasmus, Tyler DP Brunet, and Eyal Fisher.  What is interpretability?  Philosophy & Technology, 34(4):833–862, 2021.
  * [191] Duncan Pritchard.  Knowing the answer, understanding and epistemic value.  Citeseer, 2008.
  * [192] Linda Zagzebski.  On Epistemology.  Wadsworth, 2009.
  * [193] Tania Lombrozo, Daniel Wilkenfeld, T Lombrozo, and D Wilkenfeld.  Mechanistic versus functional understanding.  Varieties of understanding: New perspectives from philosophy, psychology, and theology, pages 209–229, 2019.
  * [194] Emily Sullivan.  Understanding from machine learning models.  The British Journal for the Philosophy of Science, 2022.
  * [195] Kathleen A Creel.  Transparency in complex computational systems.  Philosophy of Science, 87(4):568–589, 2020.
  * [196] Juan M Durán.  Dissecting scientific explanation in ai (sxai): A case for medicine and healthcare.  Artificial Intelligence, 297:103498, 2021.
  * [197] Carlos Zednik.  Solving the black box problem: A normative framework for explainable artificial intelligence.  Philosophy & technology, 34(2):265–288, 2021.
  * [198] Will Fleisher.  Understanding, idealization, and explainable ai.  Episteme, 19(4):534–560, 2022.
  * [199] Paulo Pirozelli.  Sources of understanding in supervised machine learning models.  Philosophy & Technology, 35(2):23, 2022.
  * [200] Maartje MA De Graaf and Bertram F Malle.  How people explain action (and autonomous intelligent systems should too).  In 2017 AAAI Fall Symposium Series, pages 19–26, 2017.
  * [201] Brent Mittelstadt, Chris Russell, and Sandra Wachter.  Explaining explanations in ai.  In Proceedings of the conference on fairness, accountability, and transparency, pages 279–288, 2019.
  * [202] Roberto Confalonieri, Tillman Weyde, Tarek R. Besold, and Fermín Moscoso del Prado Martín.  Using ontologies to enhance human understandability of global post-hoc explanations of black-box models.  Artificial Intelligence, 296, 2021.
  * [203] Ian E Nielsen, Dimah Dera, Ghulam Rasool, Ravi P Ramachandran, and Nidhal Carla Bouaynaya.  Robust explainability: A tutorial on gradient-based attribution methods for deep neural networks.  IEEE Signal Processing Magazine, 39(4):73–84, 2022.
  * [204] Jinyang Yuan, Tonglin Chen, Bin Li, and Xiangyang Xue.  Compositional scene representation learning via reconstruction: A survey.  IEEE Trans. Pattern Anal. Mach. Intell., 45(10):11540–11560, 2023.
  * [205] Tim Klinger, Dhaval Adjodah, Vincent Marois, Josh Joseph, Matthew Riemer, Alex’Sandy’ Pentland, and Murray Campbell.  A study of compositional generalization in neural models.  arXiv preprint arXiv:2006.09437, 2020.
  * [206] Lucas Rizzo and Luca Longo.  An empirical evaluation of the inferential capacity of defeasible argumentation, non-monotonic fuzzy reasoning and expert systems.  Expert Systems with Applications, 147:113220, 2020.
  * [207] Lucas Rizzo and Luca Longo.  A qualitative investigation of the explainability of defeasible argumentation and non-monotonic fuzzy reasoning.  In Proceedings for the 26th AIAI Irish Conference on Artificial Intelligence and Cognitive Science Trinity College Dublin, Dublin, Ireland, December 6-7th, 2018., pages 138–149, 2018.
  * [208] Giulia Vilone and Luca Longo.  A novel human-centred evaluation approach and an argument-based method for explainable artificial intelligence.  In Artificial Intelligence Applications and Innovations: 18th IFIP WG 12.5 International Conference, AIAI 2022, Hersonissos, Crete, Greece, June 17–20, 2022, Proceedings, Part I, pages 447–460. Springer, 2022.
  * [209] Giulia Vilone and Luca Longo.  An xai method for the automatic formation of an abstract argumentation framework from a neural network and its objective evaluation.  In 1st International Workshop on Argumentation for eXplainable AI co-located with 9th International Conference on Computational Models of Argument (COMMA 2022), volume 3209 of CEUR Workshop Proceedings. CEUR-WS.org, 2022.
  * [210] Johanna Vielhaben, Sebastian Lapuschkin, Grégoire Montavon, and Wojciech Samek.  Explainable ai for time series via virtual inspection layers.  arXiv preprint arXiv:2303.06365, 2023.
  * [211] Taufique Ahmed and Luca Longo.  Interpreting disentangled representations of person-specific convolutional variational autoencoders of spatially preserving eeg topographic maps via clustering and visual plausibility.  Information, 14(9), 2023.
  * [212] W. V. Quine.  On what there is.  In W. V. Quine, editor, From a Logical Point of View, pages 1–19. Cambridge, Mass.: Harvard University Press, 1953.
  * [213] David Krakauer.  The Computational Systems of the World.  BioScience, 64(4):351–354, April 2014.
  * [214] Samy Badreddine, Artur d’Avila Garcez, Luciano Serafini, and Michael Spranger.  Logic tensor networks.  Artificial Intelligence, 303:103649, 2022.
  * [215] Alexandre Heuillet, Fabien Couthouis, and Natalia Díaz-Rodríguez.  Explainability in deep reinforcement learning.  Knowledge-Based Systems, 214:106685, 2021.
  * [216] C Meske, E Bunde, J Schneider, and M Gersch.  Explainable artificial intelligence: Objectives, stakeholders and future research opportunities.  Information Systems Management, 2020.
  * [217] Adrian Weller.  Transparency: Motivations and Challenges.  In Wojciech Samek, Grégoire Montavon, Andrea Vedaldi, Lars Kai Hansen, and Klaus-Robert Müller, editors, Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Lecture Notes in Computer Science, pages 23–40. Springer International Publishing, Cham, 2019.
  * [218] Ronan Hamon, Henrik Junklewitz, Gianclaudio Malgieri, Paul De Hert, Laurent Beslay, and Ignacio Sanchez.  Impossible explanations? beyond explainable ai in the gdpr from a covid-19 use case scenario.  In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 549–559, 2021.
  * [219] Jessica C. Flack.  Multiple time-scales and the developmental dynamics of social systems.  Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1597):1802–1810, July 2012.  Publisher: Royal Society.
  * [220] Mislav Juric, Agneza Sandic, and Mario Brcic.  AI safety: state of the field through quantitative lens.  In 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO), pages 1254–1259, September 2020.  ISSN: 2623-8764.
  * [221] Finale Doshi-Velez and Been Kim.  Towards a rigorous science of interpretable machine learning, 2017.
  * [222] Sander Beckers.  Causal explanations and xai.  In Conference on Causal Learning and Reasoning, pages 90–109. PMLR, 2022.
  * [223] Yu-Liang Chou, Catarina Moreira, Peter Bruza, Chun Ouyang, and Joaquim Jorge.  Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications.  Information Fusion, 81:59–83, 2022.
  * [224] Martina Cinquini and Riccardo Guidotti.  Calime: Causality-aware local interpretable model-agnostic explanations.  arXiv preprint arXiv:2212.05256, 2022.
  * [225] Javier Del Ser, Alejandro Barredo-Arrieta, Natalia Díaz-Rodríguez, Francisco Herrera, and Andreas Holzinger.  Exploring the trade-off between plausibility, change intensity and adversarial power in counterfactual explanations using multi-objective optimization.  arXiv preprint arXiv:2205.10232, 2022.
  * [226] Pedro Sanchez and Sotirios A Tsaftaris.  Diffusion causal models for counterfactual estimation.  Conference on Causal Learning and Reasoning (CLeaR), 2022.
  * [227] Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein.  Diffusion visual counterfactual explanations.  In NeurIPS, 2022.
  * [228] Johannes Schneider and Michalis Vlachos.  Personalization of deep learning.  In Data Science–Analytics and Applications: Proceedings of the 3rd International Data Science Conference–iDSC2020, pages 89–96. Springer, 2021.
  * [229] Johannes Schneider and Joshua Peter Handali.  Personalized explanation for machine learning: a conceptualization.  In European Conference on Information Systems (ECIS), 2019.
  * [230] Banghua Zhu, Michael Jordan, and Jiantao Jiao.  Principled reinforcement learning with human feedback from pairwise or k-wise comparisons.  In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 43037–43067. PMLR, 23–29 Jul 2023.
  * [231] Tom Bewley and Freddy Lecue.  Interpretable preference-based reinforcement learning with tree-structured reward functions.  In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’22, page 118–126. International Foundation for Autonomous Agents and Multiagent Systems, 2022.
  * [232] Andrea Bunt, Matthew Lount, and Catherine Lauzon.  Are explanations always important?: a study of deployed, low-cost intelligent interactive systems.  In Proceedings of the 2012 ACM international conference on Intelligent User Interfaces, IUI ’12, pages 169–178, New York, NY, USA, 2012. ACM.
  * [233] Cynthia Rudin.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  Nature Machine Intelligence, 1(5):206–215, May 2019.  Number: 5 Publisher: Nature Publishing Group.
  * [234] Ronan Hamon, Henrik Junklewitz, Ignacio Sanchez, Gianclaudio Malgieri, and Paul De Hert.  Bridging the gap between ai and explainability in the gdpr: Towards trustworthiness-by-design in automated decision-makin.  IEEE Computational Intelligence Magazine, 17(1):72–85, 2022.
  * [235] David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang.  XAI—Explainable artificial intelligence.  Science Robotics, 4(37), December 2019.
  * [236] Agneza Krajna, Mario Brcic, Tomislav Lipic, and Juraj Doncevic.  Explainability in reinforcement learning: perspective and position, March 2022.
  * [237] Marzyeh Ghassemi, Luke Oakden-Rayner, and Andrew L Beam.  The false hope of current approaches to explainable artificial intelligence in health care.  The Lancet Digital Health, 3(11):e745–e750, 2021.
  * [238] Federico Cabitza, Andrea Campagner, Lorenzo Famiglini, Enrico Gallazzi, and Giovanni Andrea La Maida.  Color shadows (part i): Exploratory usability evaluation of activation maps in radiological machine learning.  In Machine Learning and Knowledge Extraction: 6th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2022, Vienna, Austria, August 23–26, 2022, Proceedings, pages 31–50. Springer, 2022.
  * [239] Federico Cabitza, Andrea Campagner, Luca Ronzio, Matteo Cameli, Giulia Elena Mandoli, Maria Concetta Pastore, Luca Sconfienza, Duarte Folgado, Marília Barandas, and Hugo Gamboa.  Rams, hounds and white boxes: Investigating human-ai collaboration protocols in medical diagnosis.  Artificial Intelligence in Medicine, page 102506, 2023.
  * [240] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld.  Does the whole exceed its parts? the effect of ai explanations on complementary team performance.  In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2021.
  * [241] Simone Natale.  Deceitful media: Artificial intelligence and social life after the Turing test.  Oxford University Press, USA, 2021.
  * [242] Federico Cabitza, Andrea Campagner, and Edoardo Datteri.  To err is (only) human. reflections on how to move from accuracy to trust for medical ai.  In Exploring Innovation in a Digital World: Cultural and Organizational Challenges, pages 36–49. Springer, 2021.
  * [243] Federico Cabitza, Andrea Campagner, and Carla Simone.  The need to move away from agential-ai: Empirical investigations, useful concepts and open issues.  International Journal of Human-Computer Studies, 155:102696, 2021.
  * [244] Tim Miller.  Explainable ai is dead, long live explainable ai! hypothesis-driven decision support using evaluative ai.  In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 333–342, 2023.
  * [245] Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli.  Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda.  In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pages 582:1–582:18, New York, NY, USA, 2018. ACM.
  * [246] Kevin Baum, Susanne Mantel, Eva Schmidt, and Timo Speith.  From responsibility to reason-giving explainable artificial intelligence.  Philosophy & Technology, 35(1):1–30, 2022.
  * [247] Stephen Thornton.  Karl Popper.  In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2023 edition, 2023.
  * [248] Matthew L. Leavitt and Ari Morcos.  Towards falsifiable interpretability research.  NeurIPS 2020 Workshop: ML Retrospectives, Surveys and Meta-Analyses (ML-RSA), 2020.
  * [249] Filip Karlo Dosilovic, Mario Brcic, and Nikica Hlupic.  Explainable artificial intelligence: A survey.  In 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pages 0210–0215, May 2018.
  * [250] Agneza Krajna, Mario Brcic, Mihael Kovac, and Ana Sarcevic.  Explainable Artificial Intelligence: An Updated Perspective.  In Proceedings of 45th Jubilee International Convention on Information, Communication and Electronic Technology (MIPRO) 2022, pages 859–864, Opatija, Croatia, May 2022.
  * [251] W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu.  Definitions, methods, and applications in interpretable machine learning.  Proceedings of the National Academy of Sciences, 116(44):22071–22080, October 2019.  Publisher: Proceedings of the National Academy of Sciences.
  * [252] Johannes Schneider, Christian Meske, and Michalis Vlachos.  Deceptive AI explanations: Creation and detection.  In International Conference on Agents and Artificial Intelligence(ICAART), pages 44–55, 2022.
  * [253] Johannes Schneider and Frank Breitinger.  AI Forensics: Did the artificial intelligence system do it? why?  arXiv preprint arXiv:2005.13635, 2020.
  * [254] Johannes Schneider and Michalis Vlachos.  Reflective-net: Learning from explanations.  Data Mining and Knowledge Discovery, 2023.
  * [255] Roman V. Yampolskiy.  Unexplainability and Incomprehensibility of AI.  Journal of Artificial Intelligence and Consciousness, 07(02):277–291, September 2020.  Publisher: World Scientific Publishing Co.
  * [256] Roman V. Yampolskiy.  What are the ultimate limits to computational techniques: verifier theory and unverifiability.  Physica Scripta, 92(9):093001, July 2017.  Publisher: IOP Publishing.
  * [257] Victor Boutin, Thomas Fel, Lakshya Singhal, Rishav Mukherji, Akash Nagaraj, Julien Colin, and Thomas Serre.  Diffusion models as artists: Are we closing the gap between humans and machines?  In International Conference on Machine Learning, 2023.
  * [258] H Holden Thorp.  Chatgpt is fun, but not an author, 2023.
  * [259] Eva AM van Dis, Johan Bollen, Willem Zuidema, Robert van Rooij, and Claudi L Bockting.  Chatgpt: five priorities for research.  Nature, 614(7947):224–226, 2023.
  * [260] Franziska Boenisch.  A systematic review on model watermarking for neural networks.  Frontiers in big Data, 4:729663, 2021.
  * [261] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein.  A watermark for large language models.  In Proceedings of the 40th International Conference on Machine Learning, pages 202:17061–17084, 2023.
  * [262] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot.  Machine unlearning.  In 2021 IEEE Symposium on Security and Privacy (SP), pages 141–159. IEEE, 2021.
  * [263] Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen.  A survey of machine unlearning.  arXiv preprint arXiv:2209.02299, 2022.
  * [264] Julie E Cohen.  Between truth and power.  Oxford University Press, 2019.
  * [265] Federico Cabitza, Andrea Campagner, Gianclaudio Malgieri, Chiara Natali, David Schneeberger, Karl Stoeger, and Andreas Holzinger.  Quod erat demonstrandum?-towards a typology of the concept of explanation for the design of explainable ai.  Expert Systems with Applications, 213:118888, 2023.
  * [266] Gianclaudio Malgieri.  “just” algorithms: justification (beyond explanation) of automated decisions under the general data protection regulation.  Law and Business, 1(1):16–28, 2021.
  * [267] Emre Bayamlioglu.  Contesting automated decisions.  Eur. Data Prot. L. Rev., 4:433, 2018.
  * [268] Clément Henin and Daniel Le Métayer.  Beyond explainability: justifiability and contestability of algorithmic decision systems.  AI & SOCIETY, pages 1–14, 2021.
  * [269] Clément Henin and Daniel Le Métayer.  A framework to contest and justify algorithmic decisions.  AI and Ethics, 1(4):463–476, 2021.
  * [270] Lisa M Austin.  Enough about me: why privacy is about power, not consent (or harm).  Forthcoming in Austin Sarat, ed., A World Without Privacy, 2014.
  * [271] Louise Wilsdon.  Carissa véliz, privacy is power: Why and how you should take back control of your data, 2022.
  * [272] Sasha Costanza-Chock.  Design justice: Community-led practices to build the worlds we need.  The MIT Press, 2020.
  * [273] Margot E Kaminski and Gianclaudio Malgieri.  Algorithmic impact assessments under the gdpr: producing multi-layered explanations.  International data privacy law, pages 19–28, 2020.
  * [274] Judith Gregory.  Scandinavian approaches to participatory design.  International Journal of Engineering Education, 19(1):62–74, 2003.
  * [275] Alessandro Mantelero.  Beyond Data: Human Rights, Ethical and Social Impact Assessment in AI.  Springer Nature, 2022.
  * [276] Gianclaudio Malgieri.  In/acceptable marketing and consumers’ privacy expectations: Four tests from eu data protection law.  Journal of Consumer Marketing, 40(2):209–223, 2023.
  * [277] Keld Bodker, Finn Kensing, and Jesper Simonsen.  Participatory IT design: designing for business and workplace realities.  MIT press, 2009.

  *[AI]: Artificial Intelligence
  *[]:
  *[XAI]: Explainable
  *[ML]: Machine Learning
  *[DL]: Deep Learning
  *[LIME]: Local Interpretable Model-Agnostic Explanations
  *[SHAP]: Shapley Additive Explanation
  *[LRP]: Layer-Wise Relevance Propagation
  *[DTs]: Decision Tree
  *[DT]: Decision Tree
  *[AIED]:  in Education
  *[LLM]: Large Language Model
  *[AIs]: Artificial Intelligence
  *[LLMs]: Large Language Model
  *[DNN]: Deep Neural Network


3, 1 XX/YY

# Tree-GPT: Modular Large Language Model Expert System for Forest Remote
Sensing Image Understanding and Interactive Analysis

S. Q. Dua,b,c S.J. Tanga,b,c Corresponding author W. X. Wang a,b,c X. M.
Lia,b,c R. Z. Guoa,b,c a Research Institute for Smart Cities, School of
Architecture and Urban Planning, Shenzhen University, Shenzhen, P.R. China
b State Key Laboratory of Subtropical Building and Urban Science, Shenzhen,
P.R. China
c Guangdong–Hong Kong-Macau Joint Laboratory for Smart Cities, P.R. China

###### Abstract

This paper introduces a novel framework, Tree-GPT, which incorporates Large
Language Models (LLMs) into the forestry remote sensing data workflow, thereby
enhancing the efficiency of data analysis. Currently, LLMs are unable to
extract or comprehend information from images and may generate inaccurate text
due to a lack of domain knowledge, limiting their use in forestry data
analysis. To address this issue, we propose a modular LLM expert system, Tree-
GPT, that integrates image understanding modules, domain knowledge bases, and
toolchains. This empowers LLMs with the ability to comprehend images, acquire
accurate knowledge, generate code, and perform data analysis in a local
environment. Specifically, the image understanding module extracts structured
information from forest remote sensing images by utilizing automatic or
interactive generation of prompts to guide the Segment Anything Model (SAM) in
generating and selecting optimal tree segmentation results. The system then
calculates tree structural parameters based on these results and stores them
in a database. Upon receiving a specific natural language instruction, the LLM
generates code based on a thought chain to accomplish the analysis task. The
code is then executed by an LLM agent in a local environment and . For
ecological parameter calculations, the system retrieves the corresponding
knowledge from the knowledge base and inputs it into the LLM to guide the
generation of accurate code. We tested this system on several tasks, including
Search, Visualization, and Machine Learning Analysis. The prototype system
performed well, demonstrating the potential for dynamic usage of LLMs in
forestry research and environmental sciences.

###### keywords:

Remote Sensing, Deep Learning, Forestry, Large Language Model, Individual Tree
Segmentation, Tree Factor Estimation, Segment Anything Model.

##  1 Introduction

Analysis of forest remote sensing data is essential for various applications
in forestry and environmental sciences[Turner et al., 2006, Suratno et al.,
2009, Indirabai et al., 2019, Sun et al., 2019]. The analysis of forest remote
sensing data is often subject to temporal constraints, necessitating the
exploration of more efficient method. Recent advancements in artificial
intelligence have given rise to Large Language Models (LLMs), which are
machine learning algorithms capable of comprehending human natural language
instruction and generating coherent text [Zhao et al., 2023]. By utilizing
natural language as an interface, researchers can employ LLMs to generate data
analysis code in a matter of minutes, a task that previously required several
tens of minutes to complete. Consequently, the integration of LLMs into the
analysis of forest remote sensing data is being considered as a means of
enhancing efficiency in a similar manner.

LMMs like GPT series have left a profound impression due to their powerful
intelligence [Ouyang et al., 2022]. Naturally, we are curious about whether we
can employ similar models to perform intelligent analysis of forest remote
sensing data. However, the limited ability of LLMs to comprehend images and
generate accurate domain-specific knowledge hinders their application in
forest remote sensing data analysis. Vision serves as the primary source of
information for human cognition. However, the majority of language models
struggle to comprehend visual information, limiting their applicability. In
recent years, researchers have investigated methods to enable LLMs to process
multimodal information. Among these studies, GPT-4 has emerged as the leading
model, exhibiting remarkable performance in understanding and accurately
describing natural images [OpenAI, 2023]. Additionally, open-source models
such as MiniGPT4 have proposed their own solutions [Zhu et al., 2023].
However, these approaches exhibit suboptimal performance on remote sensing
imagery due to the fact that the majority of these models are trained on
natural images. To address this limitation, researchers have proposed
multimodal remote sensing datasets and trained models that capable of
understanding remote sensing images and performing natural language-based
question answering [Lobry et al., 2020, Chappuis et al., 2022]. These models
can understand natural language and provide answers regarding the presence of
specific concepts in the images. However, due to their complexity, these model
are still not capable of precise quantitative analysis or executing data
analysis tasks through code generation. Furthermore, most language models
exhibit limited performance when generating text in specific professional
domains due to a lack of fine-tuning for those domains. A phenomenon known as
“hallucination” is also commonly observed in language models, where they
generate information that does not exist in reality, such as providing
incorrect interpretations of natural laws [Ji et al., 2023]. Consequently, the
lack of remote sensing image understanding ability and accurate professional
knowledge are significant issues that restrict the application of general
language models in specialized domains.

To address the first issue and enable LLMs to understand forest remote sensing
images, we are considering the integration of a separate image understanding
module. This module is built to convert the implicit information contained in
the image into explicit, structured information that LLMs can understand. For
forest remote sensing data analysis, it typically involves two primary tasks:
the individual tree crown (ITC) segmentation and calculation of tree
structural parameters. Among them, the ITC segmentation remains a challenging
problem. Current research has explored various methods based on point clouds
or images for ITC segmentation. Among them, image-based segmentation methods
tend to employ supervised learning algorithms like Mask R-CNN for accurate
instance segmentation[Beloiu et al., 2023]. These methods require a large
number of high-precision training samples. To reduce the cost of our approach,
we are considering whether unsupervised or interactive methods can be used for
ITC segmentation to generate structured information that LLMs can comprehend.

Recently, image representation learning [Vaswani et al., 2017, Dosovitskiy et
al., 2020] and self-supervised learning [He et al., 2022] have advanced
significantly, along with the availability of large-scale data and
computational resources [Kirillov et al., 2023]. These developments have
enabled the learning of universal features that can be transferred across
different domains from massive image collections. In this context, a Large
Vision Model named Segment Anything Model (SAM) was proposed [Kirillov et al.,
2023], which can perform zero-shot image segmentation guided by prompts.
However, SAM still faces two challenges in individual tree segmentation task:
1) the segmentation quality of SAM is highly sensitive to the input prompt,
and the segmentation granularity is not controllable without prior knowledge.
2) SAM generates many redundant masks in its segmentation output, which
hinders the identification of individual trees. This paper aims to overcome
these challenges and leverage the potential of SAM for single-tree
segmentation tasks by proposing a framework that does not require retraining
and can be easily adapted to new scenarios.

To address the second issue of empowering LLMs with domain-specific knowledge
and task execution capabilities, we are considering building a specialized
knowledge base and an execution agent for LLMs. The specialized knowledge base
is a widely used solution to tackle the issue of hallucination in LLMs.
Meanwhile, the execution agent acts as a framework that connects LLMs with the
local execution environment. LLMs can utilize the execution agent to execute
generated code in the local development environment, perform data
manipulation, and generate visualization or data analysis results. Currently,
there are existing frameworks for knowledge base construction and task
execution for LLMs. However, designing the optimal framework is still an open
question. For the specific task of forest remote sensing data analysis, it is
worth discussing how to adapt the existing frameworks, such as Prompt
Engineering, to suit the characteristics of this particular task.

Following above ideas, we propose a novel framework, Tree-GPT. The purpose of
Tree-GPT is to build an expert system capable of understanding forest remote
sensing images, possessing domain-specific knowledge in forest ecology, and
generating code based on prompts to automatically execute data analysis tasks.
The core of this system consists of an LLM as the reasoning engine, an image
understanding module to convert image information into text, a Domain
Knowledge Base to store professional knowledge for LLM retrieval, and an LLM
execution agent for code execution. More specifically:

  * •

LLM: The LLM and its accompanying backend execution framework are pivotal
components of Tree-GPT. In terms of model selection, the study utilizes the
cloud-deployed OpenAI GPT-4 model, known for its high performance. However, in
theory, an LLM deployed locally would also be suitable.

  * •

Image Understanding: As the second key module of Tree-GPT, the image
understanding module encompasses two functions: tree segmentation and
calculation of tree structural parameters. To enable low-cost algorithm
transfer, the study employs the Structure-Aware Masking (SAM) as the core and
designs a new tree segmentation method to address the challenges associated
with SAM. This method starts by employing a pre-trained tree detection model
to locate and outline trees (or uses an interactive approach) to generate an
initial prompt. Subsequently, the study inputs the initial prompt and a grid
prompt into SAM, resulting in redundant tree segmentation outcomes. To refine
the segmentation results, a bipartite graph matching model is utilized,
leveraging the initial prompt as a guide. After obtaining the segmentation
results, additional remote sensing data is incorporated to compute tree
structural parameters, which are stored in a relational database for
subsequent data analysis access.

  * •

Domain Knowledge Base: As the third key module of Tree-GPT, the Domain
Knowledge Base is a vector database that stores forest ecology domain-specific
knowledge in embedded text form. This knowledge base allows for retrieval of
crucial information through keyword searches and utilizes it as prompts to
guide the LLM in generating accurate professional knowledge outputs.

  * •

LLM Execution Agent: Lastly, as the execution module of Tree-GPT, the LLM
execution agent serves as a backend framework that connects the LLM’s code
outputs with the local runtime environment. The study has tailored the Prompt
Engineering framework to address the specific tasks of forest remote sensing
data analysis, including information retrieval, result visualization, and data
analysis. Task-oriented prompt design ensures the generation of precise code,
facilitating efficient and effective data analysis.

Figure 1: Overall Framework of the Tree-GPT.

In conclusion, Tree-GPT offers a user-friendly approach that enables real-time
data updates and analysis. This reduces the time required for the processing
workflow and allows researchers to allocate more resources towards data
interpretation and the understanding of ecological patterns. With its
understanding and generation capabilities provided by Large Language Models,
Tree-GPT offers a pertinent contribution to the field of tree ecological
parameter extraction.

##  2 Related Works

###  2.1 Individual Tree Crown (ITC) Segmentation

Efficient and automated ITC segmentation methods are crucial for calculating
tree factors [Yang et al., 2019]. Existing methods for ITC segmentation can be
classified based on the type of data used, including point cloud based methods
and image based method. Current UAV point cloud based methods often utilize
unsupervised clustering techniques. Since the highest point of a tree (which
can be regarded as the tree’s center) is easily identifiable in the point
cloud, these methods typically determine the tree center by leveraging the
elevation differences. They then perform point cloud clustering using the
center as a seed point to segment the tree contour [Strîmbu and Strîmbu,
2015]. However, accurately describing tree boundaries becomes challenging due
to the lack of texture information in point clouds. As a result, most methods
rely on strong prior assumptions such as the conical shape assumption to
differentiate between points at the boundaries of trees [Strîmbu and Strîmbu,
2015, Yang et al., 2020, Qian et al., 2023]. These methods generally achieve
acceptable results in most scenarios. However, in subtropical or tropical
regions, trees tend to have broader and more expansive shapes with large crown
areas [Lutz et al., 2012]. As a result, many trees’ growth patterns deviate
from the aforementioned prior assumptions, which limits the generalization
capability of these methods when applied to new scenarios, necessitating
repeated parameter tuning.

Based on image data, tree segmentation methods primarily utilize the color and
texture differences of different trees in RGB images or the tree height
differences in the canopy height model (CHM) to segment individual trees. Some
studies have employed traditional morphological image segmentation methods,
such as watershed algorithms and their variants, to achieve tree segmentation
in RGB or CHM images [Tochon et al., 2015, Wagner et al., 2018, Yang et al.,
2019]. However, the limited utilization of morphological image features
hinders the accuracy of these segmentation methods. In recent years, research
has focused on deep learning-based single-tree segmentation methods due to the
advancement of deep learning techniques. Compared to traditional image
segmentation methods, deep learning networks possess stronger feature
extraction capabilities and can learn features with certain generalization
properties, thus enhancing the generalization of the methods [LeCun et al.,
2015]. Currently, most deep learning-based single-tree segmentation methods
are based on the Mask-RCNN network [He et al., 2017, Beloiu et al., 2023].
Some methods have improved edge extraction by enhancing the loss function
[Zhang et al., 2022], while others have considered incorporating multi-modal
feature extraction techniques within the Mask-RCNN framework. They
simultaneously use texture, color, and elevation information from both RGB and
CHM images to improve tree segmentation accuracy [Li et al., 2022]. Although
deep learning-based instance segmentation methods for single-tree segmentation
show significant improvements over traditional image segmentation methods in
terms of robustness and generalization, most supervised deep learning methods
suffer from a fundamental limitation: their performance is heavily constrained
by the training data. Since deep learning is essentially about finding a set
of factors that best fit the distribution of the training samples, the
performance of the network is determined by the extent to which the
probability distribution of the training samples aligns with the real-world
distribution [Goodfellow et al., 2016]. For single-tree segmentation tasks,
high-precision open-source tree contour datasets are severely lacking.
Moreover, distinguishing and annotating tree contours is extremely costly
compared to natural images, making it difficult to rapidly annotate a large
amount of data to train a tree segmentation network with strong generalization
and zero-shot transfer capability.

##  3 Method

Tree-GPT is a specialized system designed to process remote sensing data from
forests. It comprises a GPT-4-based LLM, an image understanding module, a
Domain Knowledge Base, and an LLM execution agent. The Tree-GPT workflow
operates as follows: given a natural language instruction $L$ and a remote
sensing image $I$ of the study area, the image understanding module processes
$I$ and collaborates with other remote sensing data from the study area to
generate tree structure parameters that can be used as inputs for the LLM and
for data analysis. These parameters are stored in a relational database $R$.
Simultaneously, the professional domain knowledge base module assesses $L$ to
determine if it is a knowledge-based question and whether to invoke the
professional knowledge base module to answer it. If $L$ is not a knowledge-
based question, it proceeds to the LLM execution agent module.

In the LLM execution module, if $L$ does not involve code execution tasks, the
output result of the LLM is directly returned in the FastChat frontend. If the
result contains code execution tasks, the corresponding code is generated
based on prompt templates and executed using the LangChain agent module in a
local runtime environment. During execution, data visualization or analysis
results are generated by calling image $I$ and database $R$.

Figure 2: Visualization of individual tree crown(ITC) segmentation results.
The first line shows tree crown segmentation results (segmentation in 2D). The
second line shows individual tree point cloud segmentation results (3D
segmentation by projecting 2D results to point cloud).

###  3.1 Image Understanding Module

Mask Generation:The Tree-GPT model utilizes a tree target detection framework
and a bipartite graph matching method to control the segmentation granularity
of SAM. This generates tree segmentation results that correspond one-to-one
with the tree position. Specifically, the single-tree segmentation module uses
points and target frames as SAM’s Prompt at the same time to guide the
generation of tree Masks. Among them, the target detection module can use
automatic or manual methods to generate tree target detection frames, which
are used to guide SAM for tree segmentation and assign semantic information to
the segmentation results. The automatic target frame generation method uses
RetinaNet [Lin et al., 2017] as the target detection framework with ResNet50
[He et al., 2016] as the backbone network and DeepForest [Weinstein et al.,
2019] pretrained weights. Manual annotation of tree target frames can be used
as a supplement to automatic tree target detection to improve the accuracy of
tree detection in an interactive manner. In addition to the target frame, we
use a 48 $\times$ 48 grid sampling point as the basic Prompt for SAM to deal
with situations where BBox Prompt guidance fails. This generates enough Masks
for subsequent matching with the target frame and obtains optimal segmentation
results of the tree. After obtaining the Prompt, we input Point and BBox
Prompt into SAM respectively and merge the two sets of tree segmentation
results to obtain redundant tree segmentation results.

Best Mask Matching: After obtaining the redundant tree segmentation results,
we consider how to select the optimal tree segmentation results that
correspond one-to-one with the real trees. Since SAM’s segmentation results do
not contain semantic information, we consider using the automatically or
manually obtained tree target detection results as the real position of the
tree. We match the redundant Mask with the target detection results to obtain
the tree segmentation results corresponding to the tree position.
Specifically, we model this matching problem as a bipartite graph matching
problem in mathematics. Bipartite graph matching, also known as bipartite
matching, is a model in graph theory where given two disjoint sets of vertices
A and B, and a set of edges $E$ $\subseteq$ $A$ $\times$ $B$, a matching $M$
is a subset of $E$ such that no two edges share an endpoint. A maximum
matching is a matching $M$ of maximum cardinality (maximum number of edges).
This problem can be defined as:

$\text{min:}\sum_{i\in A}\sum_{j\in B}c_{ij}x_{ij}$ (1) s.t.
$\displaystyle\sum_{i\in A}x_{ij}=1,\text{ for all }j\in B$ (2)
$\displaystyle\sum_{j\in B}x_{ij}=1,\text{ for all }i\in A$ $\displaystyle
x_{ij}\in\\{0,1\\},\text{ for all }(i,j)\in E$

Where the matchings are represented by variables $x_{ij}$ for each edge
$E(i,j)$. $c_{ij}$ is the cost of matching vertex i to vertex j in the
bipartite graph. The constraints ensure that each vertex is matched exactly
once.

In short, the bipartite graph matching problem is used to solve a set of
elements in set $B$ that correspond one-to-one with elements in set $A$. This
correspondence is called maximum matching. When solving bipartite graph
matching, it is necessary to define the cost of adding each edge, that is, the
Cost matrix. In this article, we aim to maximize the overlap between the
segmentation result and our detected target frame. We can use GIoU
[Rezatofighi et al., 2019] to characterize this degree of overlap. The GIoU is
defined as IoU minus the area of the smallest enclosing box that covers both
bounding boxes P and G, excluding the union area of P and G and can be
expressed as follows:

$IoU=\frac{{\text{{Area of Intersection}}}}{{\text{{Area of Union}}}}$ (3)

$GIoU=IoU-\frac{A_{c}-A_{u}}{A_{c}}$ (4)

After obtaining the Cost matrix, we use the classic Hungarian algorithm to
solve bipartite graph matching and obtain the optimal segmentation result with
the highest overlap with tree position.

Tree Structure Factor Estimation: After obtaining the tree segmentation
results, we use the 2D segmentation results to assign labels to the point
cloud within the corresponding range. This allows us to obtain the 3D single-
tree segmentation results. After obtaining the 3D segmentation, we calculate
the tree height, crown width, support height and crown area. We store them in
the tree database together with the 2D segmentation results, external target
frame and tree position information. Among them, the tree position is stored
in the database in pixel coordinates. The 2D tree contour is stored in the
database in MS COCO’s [Lin et al., 2014] compressed storage format.

###  3.2 Domain Knowledge Base

The Domain Knowledge Base is a vector database that stores embedded text of
ecological knowledge in the field of forestry. The module first uses the
OpenAIEmbedding API to embed the text of ecological knowledge in the forestry
field and convert it into numerical vectors. The text is then segmented using
the LangChain framework’s text segmentation tool with a block size of 4,000
tokens, and the resulting knowledge vectors are split into several blocks and
stored in the Chroma vector database. When Tree-GPT receives a natural
language instruction, it is first embedded using the OpenAIEmbedding API and
converted into a numerical vector. The module then uses the Facebook AI
Similarity Search (FAISS) algorithm to retrieve relevant information from the
Chroma database, obtaining retrieval results denoted as SS. These results are
ranked, and text blocks with a similarity greater than 0.6 are selected. These
selected text blocks are then used as new inputs to the LLM using the prompt
template ’Given context, could you please explain the meaning of query?’ to
obtain the final inference result. This allows the model to accurately answer
knowledge-based questions in the professional domain.

###  3.3 LLM Execution Agent

The LLM Execution Agent is a backend framework in TreeGPT that bridges the
LLM’s generated code structure with the local runtime environment. Built on
the LangChain toolchain, the LLM Execution Agent executes instructions in two
steps: Task Planning and Code Execution. In Task Planning, the LLM decomposes
natural language instructions into subtasks using prompt templates and
generates corresponding content for each subtask. By breaking down tasks, the
LLM is more likely to generate accurate code, as the subtasks are simpler.
TreeGPT employs task planning templates, as shown in the Table 1.

| Prompt
---
Templete
Input | Output
1\. Thought |  | Natural Language
---
Instruction
| Task
---
Decomposition
2\. Action |  | Task
---
Decomposition
| Type of
---
Sub-Tasks
3\. Action Input | Sub-Tasks | Code
4\. Observation | Code |  | Running
---
Results
5\. Thought |  | Running Results +
---
Instruction
Output Text
6\. Final Result | - |  | Output Text+
---
Running Results
(Visualization, etc.)
Table 1: Process of LLM Execution Agent.

##  4 Experiment Results

Figure 3: The results of Tree-GPT’s operation.

We use data collected from Shenzhen, Guangdong Province, China to verify the
effectiveness of our method. The data was collected in 2018, with an RGB
orthoimage spatial resolution of 0.025m and a Lidar resolution of 100 points
per $m^{2}$. In the experiment, we cropped three 3000$\times$3000 pixel areas
to verify the effectiveness of the method. In terms of language model
selection, we use the ChatGPT API to access the database to maximize the
accuracy of code generation.

Our experiments on Tree-GPT revolve around two tasks: tree segmentation and
data visualization and analysis based on natural language. The results are
presented in Figure 2 and 3. As shown in Figure 2, in the tree segmentation
task, Tree-GPT’s segmentation results are basically consistent with manual
annotations. It can even be said that in some cases the manual annotation
results may not be as accurate as SAM’s results. In addition, based on natural
language-guided tree parameter query, visualization and analysis results are
presented in Figures 3 respectively. The operating results of Tree-GPT are
demonstrated in various tasks, including simple visualization (directly
generating scatter plots), information retrieval (finding the tallest tree),
complex visualization (drawing a tree growth diagram based on the position and
crown width of the tree and generating a box plot of trees grouped by height),
code generation, and statistical learning-based analysis and testing of tree
ecological parameters (estimating the Gaussian distribution parameters of tree
height and using RMSE as a confidence measure). As shown in the figure3,
during the query, Tree-GPT can give correct query results and visualize them
in the figure. In the data visualization task, Tree-GPT can give basically
correct results in the first round. However, it may require multiple manual
guidance to achieve the best visualization effect. In the data analysis task,
Tree-GPT can implement simple machine learning code for various tasks.

##  5 Conclusion

In conclusion, Tree-GPT offers a user-friendly approach that enables real-time
data updates and analysis, thereby reducing the time required for the
processing workflow and allowing researchers to allocate more resources
towards data interpretation and the understanding of ecological patterns. With
its understanding and generation capabilities provided by Large Language
Models, Tree-GPT offers a pertinent contribution to the field of tree
ecological parameter extraction.

## ACKNOWLEDGEMENTS

This work was supported in part by the National Key Research and Development
Program of China (Project No. 2022 YFB 3903700), a Shenzhen Science and
Technology Program (Project No. JCYJ 20210324093012033), the Natural Science
Foundation of Guangdong Province (Project No. 2121A1515012574), the National
Natural Science Foundation of China (Project Nos. 71901147, 41901329,
41971354, and 41971341,42001331), Shenzhen Key Laboratory of Digital Twin
Technologies for Cities (Project No: ZDSYS20210623101800001).

## References

  * [Beloiu et al., 2023] Beloiu, M., Heinzmann, L., Rehush, N., Gessler, A., Griess, V. C., 2023. Individual Tree-Crown Detection and Species Identification in Heterogeneous Forests Using Aerial RGB Imagery and Deep Learning. Remote Sensing, 15(5), 1463.
  * [Chappuis et al., 2022] Chappuis, C., Zermatten, V., Lobry, S., Le Saux, B., Tuia, D., 2022. Prompt-rsvqa: Prompting visual context to a language model for remote sensing visual question answering. _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 1372–1381.
  * [Chiang et al., 2023] Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E. et al., 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. lmsys. org (accessed 14 April 2023).
  * [Dosovitskiy et al., 2020] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S. et al., 2020\. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.
  * [Goodfellow et al., 2016] Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep learning. MIT press.
  * [He et al., 2022] He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R., 2022. Masked autoencoders are scalable vision learners. _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 16000–16009.
  * [He et al., 2017] He, K., Gkioxari, G., Dollár, P., Girshick, R., 2017. Mask r-cnn. _Proceedings of the IEEE international conference on computer vision_ , 2961–2969.
  * [He et al., 2016] He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 770–778.
  * [Indirabai et al., 2019] Indirabai, I., Nair, M. H., Nair, J. R., Nidamanuri, R. R., 2019. Estimation of forest structural attributes using ICESat/GLAS-spaceborne laser altimetry data in the western ghats region of India. Journal of Geovisualization and Spatial Analysis, 3, 1–13.
  * [Ji et al., 2023] Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., Fung, P., 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), 1–38.
  * [Kirillov et al., 2023] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y. et al., 2023. Segment anything. arXiv preprint arXiv:2304.02643.
  * [LeCun et al., 2015] LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. nature, 521(7553), 436–444.
  * [Li et al., 2022] Li, Y., Chai, G., Wang, Y., Lei, L., Zhang, X., 2022. Ace r-cnn: An attention complementary and edge detection-based instance segmentation algorithm for individual tree species identification using uav rgb images and lidar data. Remote Sensing, 14(13), 3035.
  * [Lin et al., 2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P., 2017. Focal loss for dense object detection. _Proceedings of the IEEE international conference on computer vision_ , 2980–2988.
  * [Lin et al., 2014] Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C. L., 2014. Microsoft coco: Common objects in context. _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_ , Springer, 740–755.
  * [Lobry et al., 2020] Lobry, S., Marcos, D., Murray, J., Tuia, D., 2020. RSVQA: Visual question answering for remote sensing data. IEEE Transactions on Geoscience and Remote Sensing, 58(12), 8555–8566.
  * [Lutz et al., 2012] Lutz, J. A., Larson, A. J., Swanson, M. E., Freund, J. A., 2012. Ecological Importance of Large-Diameter Trees in a Temperate Mixed-Conifer Forest. PLOS ONE, 7(5), 1-15. https://doi.org/10.1371/journal.pone.0036131.
  * [OpenAI, 2023] OpenAI, 2023. Gpt-4 technical report.
  * [Ouyang et al., 2022] Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A. et al., 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35, 27730–27744.
  * [Qian et al., 2023] Qian, C., Yao, C., Ma, H., Xu, J., Wang, J., 2023. Tree species classification using airborne lidar data based on individual tree segmentation and shape fitting. Remote Sensing, 15(2), 406.
  * [Rezatofighi et al., 2019] Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., Savarese, S., 2019\. Generalized intersection over union: A metric and a loss for bounding box regression. _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 658–666.
  * [Strîmbu and Strîmbu, 2015] Strîmbu, V. F., Strîmbu, B. M., 2015. A graph-based segmentation algorithm for tree crown extraction using airborne LiDAR data. ISPRS Journal of Photogrammetry and Remote Sensing, 104, 30–43.
  * [Sun et al., 2019] Sun, Y., Xin, Q., Huang, J., Huang, B., Zhang, H., 2019. Characterizing tree species of a tropical wetland in southern china at the individual tree level based on convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(11), 4415–4425.
  * [Suratno et al., 2009] Suratno, A., Seielstad, C., Queen, L., 2009. Tree species identification in mixed coniferous forest using airborne laser scanning. ISPRS Journal of Photogrammetry and Remote Sensing, 64(6), 683–693.
  * [Tochon et al., 2015] Tochon, G., Feret, J.-B., Valero, S., Martin, R. E., Knapp, D. E., Salembier, P., Chanussot, J., Asner, G. P., 2015. On the use of binary partition trees for the tree crown segmentation of tropical rainforest hyperspectral images. Remote sensing of environment, 159, 318–331.
  * [Turner et al., 2006] Turner, J. A., Buongiorno, J., Zhu, S., 2006. An economic model of international wood supply, forest stock and forest area change. Scandinavian Journal of Forest Research, 21(1), 73–86.
  * [Vaswani et al., 2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., Polosukhin, I., 2017. Attention is all you need. Advances in neural information processing systems, 30.
  * [Wagner et al., 2018] Wagner, F. H., Ferreira, M. P., Sanchez, A., Hirye, M. C., Zortea, M., Gloor, E., Phillips, O. L., de Souza Filho, C. R., Shimabukuro, Y. E., Aragão, L. E., 2018. Individual tree crown delineation in a highly diverse tropical forest using very high resolution satellite images. ISPRS journal of photogrammetry and remote sensing, 145, 362–377.
  * [Weinstein et al., 2019] Weinstein, B. G., Marconi, S., Bohlman, S., Zare, A., White, E., 2019. Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks. Remote Sensing, 11(11), 1309.
  * [Yang et al., 2020] Yang, J., Kang, Z., Cheng, S., Yang, Z., Akwensi, P. H., 2020. An individual tree segmentation method based on watershed algorithm and three-dimensional spatial distribution analysis from airborne LiDAR point clouds. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13, 1055–1067.
  * [Yang et al., 2019] Yang, Q., Su, Y., Jin, S., Kelly, M., Hu, T., Ma, Q., Li, Y., Song, S., Zhang, J., Xu, G. et al., 2019. The influence of vegetation characteristics on individual tree segmentation methods with airborne LiDAR data. Remote Sensing, 11(23), 2880.
  * [Zhang et al., 2023] Zhang, C., Liu, L., Cui, Y., Huang, G., Lin, W., Yang, Y., Hu, Y., 2023. A Comprehensive Survey on Segment Anything Model for Vision and Beyond. arXiv preprint arXiv:2305.08196.
  * [Zhang et al., 2022] Zhang, C., Zhou, J., Wang, H., Tan, T., Cui, M., Huang, Z., Wang, P., Zhang, L., 2022. Multi-species individual tree segmentation and identification based on improved mask R-CNN and UAV imagery in mixed forests. Remote Sensing, 14(4), 874.
  * [Zhao et al., 2023] Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z. et al., 2023. A survey of large language models. arXiv preprint arXiv:2303.18223.
  * [Zhu et al., 2023] Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M., 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592.


# On Generalized $(m,n)$-Jordan Derivations and Centralizers of Semiprime
Rings

Driss Bennis${}^{1,\textbf{a}}$, Basudeb Dhara2 and Brahim
Fahid${}^{1,\textbf{b}}$

1: Centre de Recherche de Mathématiques et Applications de Rabat (CeReMAR),
Faculty of Sciences, Mohammed V University in Rabat, Rabat, Morocco
a<EMAIL_ADDRESS><EMAIL_ADDRESS>
b<EMAIL_ADDRESS>

2: Department of Mathematics, Belda College, Belda,
Paschim Medinipur, 721424, W.B., India.
<EMAIL_ADDRESS>

Abstract. In this paper we give an affirmative answer to two conjectures on
generalized $(m,n)$-Jordan derivations and generalized $(m,n)$-Jordan
centralizers raised in [S. Ali and A. Fošner, On Generalized
$(m,n)$-Derivations and Generalized $(m,n)$-Jordan Derivations in Rings,
Algebra Colloq. 21 (2014), 411–420] and [A. Fošner, A note on generalized
$(m,n)$-Jordan centralizers, Demonstratio Math. 46 (2013), 254–262].
Precisely, when $R$ is a semiprime ring, we prove, under some suitable torsion
restrictions, that every nonzero generalized $(m,n)$-Jordan derivation (resp.,
a generalized $(m,n)$-Jordan centralizer) is a derivation (resp., a two-sided
centralizer).

2010 Mathematics Subject Classification. 16N60, 16W25

Key Words. semiprime ring, generalized $(m,n)$-derivation, generalized
$(m,n)$-Jordan derivation, $(m,n)$-Jordan centralizer, generalized
$(m,n)$-Jordan centralizer

##  1 Introduction

Throughout this paper, $R$ will represent an associative ring with center
$Z(R)$. We denote by $char(R)$ the characteristic of a prime ring $R$. Let
$n\geq 2$ be an integer. A ring $R$ is said to be $n$-torsion free if, for all
$x\in R$, $nx=0$ implies $x=0$. Recall that a ring $R$ is prime if, for any
$a,b\in R$, $aRb=\\{0\\}$ implies $a=0$ or $b=0$. A ring $R$ is called
semiprime if, for any $a\in R$, $aRa=\\{0\\}$ implies $a=0$.

An additive mapping $d:R\longrightarrow R$ is called a derivation, if
$d(xy)=d(x)y+xd(y)$ holds for all $x,y\in R$, and it is called a Jordan
derivation, if $d(x^{2})=d(x)x+xd(x)$ holds for all $x\in R$. An additive
mapping $T:R\longrightarrow R$ is called a left (resp., right) centralizer if
$T(xy)=T(x)y$ (resp., $T(xy)=xT(y)$) is fulfilled for all $x,y\in R$, and it
is called a left (resp., right) Jordan centralizer if $T(x^{2})=T(x)x$ (resp.,
$T(x^{2})=xT(x)$) is fulfilled for all $x\in R$. We call an additive mapping
$T:R\longrightarrow R$ a two-sided centralizer (resp., a two-sided Jordan
centralizer) if $T$ is both a left as well as a right centralizer (resp., a
left and a right Jordan centralizer).

An additive mapping $F:R\longrightarrow R$ is called a generalized derivation
if $F(xy)=F(x)y+xd(y)$ holds for all $x,y\in R$, where $d:R\longrightarrow R$
is a derivation. The concept of generalized derivations was introduced by
Brešar in [3] and covers both the concepts of derivations and left
centralizers. It is easy to see that generalized derivations are exactly those
additive mappings $F$ which can be written in the form $F=d+T$, where $d$ is a
derivation and $T$ is a left centralizer.

The Jordan counterpart of the notion of generalized derivation was introduced
by Jing and Lu in [10] as follows: An additive mapping $F:R\longrightarrow R$
is called a generalized Jordan derivation if $F(x^{2})=F(x)x+xd(x)$ is
fulfilled for all $x\in R$, where $d:R\longrightarrow R$ is a Jordan
derivation.

The study of relations between various sorts of derivations goes back to
Herstein’s classical result [9] which shows that any Jordan derivation on a
$2$-torsion free prime ring is a derivation (see also [5] for a brief proof of
Herstein’s result). In [7], Cusack generalized Herstein’s result to
$2$-torsion free semiprime rings (see also [2] for an alternative proof).
Motivated by these classical results, Vukman [17] proved that any generalized
Jordan derivation on a $2$-torsion free semiprime ring is a generalized
derivation.

In the last few years several authors have introduced and studied various
sorts of parameterized derivations. In [1], Ali and Fošner defined the notion
of $(m,n)$-derivations as follows: Let $m,n\geq 0$ be two fixed integers with
$m+n\neq 0$. An additive mapping $d:R\longrightarrow R$ is called an
$(m,n)$-derivation if

$(m+n)d(xy)=2md(x)y+2nxd(y)$ (1.1)

holds for all $x,y\in R$.

Obviously, a $(1,1)$-derivation on a $2$-torsion free ring is a derivation.

In the same paper [1], a generalized $(m,n)$-derivation was defined as
follows: Let $m,n\geq 0$ be two fixed integers with $m+n\neq 0$. An additive
mapping $D:R\longrightarrow R$ is called a generalized $(m,n)$-derivation if
there exists an $(m,n)$-derivation $d:R\longrightarrow R$ such that

$(m+n)D(xy)=2mD(x)y+2nxd(y)$ (1.2)

holds for all $x,y\in R$.

Obviously, every generalized $(1,1)$-derivation on a $2$-torsion free ring is
a generalized derivation.

In [18], Vukman defined an $(m,n)$-Jordan derivation as follows: Let $m,n\geq
0$ be two fixed integers with $m+n\neq 0$. An additive mapping
$d:R\longrightarrow R$ is called an $(m,n)$-Jordan derivation if

$(m+n)d(x^{2})=2md(x)x+2nxd(x)$ (1.3)

holds for all $x,y\in R$.

Clearly, every $(1,1)$-Jordan derivation on a $2$-torsion free ring is a
Jordan derivation.

Recently, in [11], Kosi-Ulbl and Vukman proved the following result.

######  Theorem 1.1 ([11], Theorem 1.5)

Let $m,n\geq 1$ be distinct integers, $R$ a $mn(m+n)|m-n|$-torsion free
semiprime ring an $d:R\longrightarrow R$ an $(m,n)$-Jordan derivation. Then
$d$ is a derivation which maps $R$ into $Z(R)$.

The $(m,n)$-generalized counterpart of the notion of an $(m,n)$-Jordan
derivation is introduced by Ali and Fošner in [1] as follows: Let $m,n\geq 0$
be two fixed integers with $m+n\neq 0$. An additive mapping
$F:R\longrightarrow R$ is called a generalized $(m,n)$-Jordan derivation if
there exists an $(m,n)$-Jordan derivation $d:R\longrightarrow R$ such that

$(m+n)F(x^{2})=2mF(x)x+2nxd(x)$ (1.4)

holds for all $x,y\in R$.

Based on some observations and inspired by the classical results, Ali and
Fošner in [1] made the following conjecture.

######  Conjecture 1.2 ([1], Conjecture 1)

Let $m,n\geq 1$ be two fixed integers, let $R$ be a semiprime ring with
suitable torsion restrictions, and let $F:R\longrightarrow R$ be a nonzero
generalized $(m,n)$-Jordan derivation. Then $F$ is a derivation which maps $R$
into $Z(R)$.

The first aim of this paper is to give an affirmative answer to this
conjecture. Namely, our first main result is the following theorem.

###### Theorem 1.3

Let $m,n\geq 1$ be distinct integers, let $R$ be a $k$-torsion free semiprime
ring, where $k=6mn(m+n)|m-n|$, and let $F:R\longrightarrow R$ be a nonzero
generalized $(m,n)$-Jordan derivation. Then $F$ is a derivation which maps $R$
into $Z(R)$.

On the other hand and in parallel, there are similar works which study
relations between various sorts of Jordan centralizers and centralizers.
Namely, in [20], Zalar proved that any left (resp., right) Jordan centralizer
on a $2$-torsion free semiprime ring is a left (resp., right) centralizer. In
[15], Vukman proved that, for a $2$-torsion free semiprime ring $R$, every
additive mapping $T:R\longrightarrow R$ satisfying the relation
“$2T(x^{2})=T(x)x+xT(x)$ for all $x\in R$” is a two-sided centralizer.
Motivated by these results and inspired by his work [15], Vukman in [19]
introduced the notion of an $(m,n)$-Jordan centralizer as follows: Let
$m,n\geq 0$ be two fixed integers with $m+n\neq 0$. An additive mapping
$T:R\longrightarrow R$ is called an $(m,n)$-Jordan centralizer if

$(m+n)T(x^{2})=mT(x)x+nxT(x)$ (1.5)

holds for all $x,y\in R$.

Obviously, a $(1,0)$-Jordan centralizer (resp., $(0,1)$-Jordan centralizer) is
a left (resp., a right) Jordan centralizer. When $n=m=1$, we recover the maps
studied in [15].

Based on some observations and results, Vukman conjectured that, on semiprime
rings with suitable torsion restrictions, every $(m,n)$-Jordan centralizer is
a two-sided centralizer (see [19]). Recently, this conjecture was solved
affirmatively by Kosi-Ulbl and Vukman in [12]. Namely, they proved the
following result.

######  Theorem 1.4 ([12], Theorem 1.5)

Let $m,n\geq 1$ be distinct integers, let $R$ be an $mn(m+n)$-torsion free
semiprime ring, and let $T:R\longrightarrow R$ be an $(m,n)$-Jordan
centralizer. Then $T$ is a two-sided centralizer.

Inspired by the work of Vukman [15, 19], Fošner [8] introduced more
generalized version of $(m,n)$-Jordan centralizers as follows: Let $m,n\geq 0$
be two fixed integers with $m+n\neq 0$. An additive mapping
$T:R\longrightarrow R$ is called a generalized $(m,n)$-Jordan centralizer if
there exists an $(m,n)$-Jordan centralizer $T_{0}:R\longrightarrow R$ such
that

$(m+n)T(x^{2})=mT(x)x+nxT_{0}(x)$ (1.6)

holds for all $x\in R$.

Thus, a generalized $(1,0)$-Jordan centralizer is a left Jordan centralizer.

In [8], Fošner showed that, on a prime ring with a specific torsion condition,
every generalized $(m,n)$-Jordan centralizer is a two-sided centralizer. This
led Fošner to make the following conjecture.

######  Conjecture 1.5 ([8], Conjecture 1)

Let $m,n\geq 1$ be two fixed integers, let $R$ be a semiprime ring with
suitable torsion restrictions, and let $T:R\longrightarrow R$ be a generalized
$(m,n)$-Jordan centralizer. Then $T$ is a two-sided centralizer.

The second aim of this paper is to give an affirmative answer to Fošner’s
conjecture. Namely, our second main result is the following theorem.

###### Theorem 1.6

Let $m,n\geq 1$ be two fixed integers, let $R$ be an $6mn(m+n)(2n+m)$-torsion
free semiprime ring, and let $T:R\longrightarrow R$ be a nonzero generalized
$(m,n)$-Jordan centralizer. Then $T$ is a two-sided centralizer.

##  2 Proof of the main theorems

In the proof of our main results, Theorems 1.3 and 1.6, we shall use the
following results.

######  Lemma 2.1 ([1], Lemma 1)

Let $m,n\geq 0$ be distinct integers with $m+n\neq 0$, let $R$ be a
$2$-torsion free ring, and let $F:R\longrightarrow R$ be a nonzero generalized
$(m,n)$-Jordan derivation with an associated $(m,n)$-Jordan derivation $d$.
Then,
$(m+n)^{2}F(xyx)=m(n-m)F(x)xy+m(3m+n)F(x)yx+m(m-n)F(y)x^{2}+4mnxd(y)x+n(n-m)x^{2}d(y)+n(m+3n)xyd(x)+n(m-n)yxd(x)$
for all $x,y\in R$.

######  Lemma 2.2 ([6], Theorem 3.3)

Let $n\geq 2$ be a fixed integer and let $R$ be a prime ring with $char(R)=0$
or $char(R)\geq n$. If $T:R\longrightarrow R$ is an additive mapping
satisfying the relation $T(x^{n})=T(x)x^{n-1}$ for all $x\in R$, then
$T(xy)=T(x)y$ for all $x,y\in R$.

######  Lemma 2.3 ([8], Lemma 1)

Let $m,n\geq 0$ be distinct integers with $m+n\neq 0$, let $R$ be a ring, and
let $T:R\longrightarrow R$ be a nonzero generalized $(m,n)$-Jordan centralizer
with an associated $(m,n)$-Jordan centralizer $T_{0}$. Then,
$2(m+n)^{2}T(xyx)=mnT(x)xy+m(2m+n)T(x)yx-
mnT(y)x^{2}+2mnxT_{0}(y)x-mnx^{2}T_{0}(y)+n(m+2n)xyT_{0}(x)+mnyxT_{0}(x)$ for
all $x,y\in R$.

######  Lemma 2.4 ([16], Lemma 3)

Let $R$ be a semiprime ring and let $T:R\longrightarrow R$ be an additive
mapping. If either $T(x)x=0$ or $xT(x)=0$ holds for all $x\in R$, then $T=0$.

We shall use the relation between semiprime rings and prime ideals. Namely, it
is well-known that a ring $R$ is semiprime if and only if the intersection of
all prime ideals of $R$ is zero if and only if $R$ has no nonzero nilpotent
(left, right) ideals (see for instance Lam’s book [13] or the recent book of
Brešar [4]). Due to the classical Levitzki’s paper [14], several authors
prefer to refer to a such result by Levitzki’s lemma.

Let $I$ be an ideal of $R$. For an element $x\in R$, we use $\overline{x}$ to
denote the equivalence class of $x$ modulo $I$.

###### Lemma 2.5

Let $R$ be both a $2$-torsion free and a $3$-torsion free semiprime ring and
let $T:R\longrightarrow R$ be an additive map such that $T(x)x^{3}=0$ and
$T(x^{4})=0$ for all $x\in R$. Then $T(xy)=T(x)y$ for all $x,y\in R$.

Proof. Let $x,y\in R$. We prove that $T(xy)=T(x)y$. We may assume that $x$ and
$y$ are not $0$. Let $P$ be a prime ideal of $R$ and set $\overline{R}=R/P$.
Consider an element $p\in P$. By hypothesis,
$0=T(x+p)(x+p)^{3}=(T(x)+T(p))(x^{3}+xpx+px^{2}+p^{2}x+x^{2}p+xp^{2}+pxp+p^{3})$.
Thus,
$0=T(x)(xpx+px^{2}+p^{2}x+x^{2}p+xp^{2}+pxp+p^{3})+T(p)(x^{3}+xpx+px^{2}+p^{2}x+x^{2}p+xp^{2}+pxp)$.
Hence, $T(p)x^{3}\in P$, equivalently $\overline{T(p)}\overline{x}^{3}=0$. By
Levitzki’s lemma, $\overline{T(p)}\overline{x}=0$, and then
$\overline{T(p)}=0$ (since $\overline{R}$ is a prime ring). Thus,
$T(P)\subseteq P$, which implies that $T(x+P)=T(x)+P$. Then, the induced map
$\overline{T}:R/P\rightarrow R/P$ such that
$\overline{T}(\overline{x})=\overline{T(x)}$ for every $x\in R$, is well
defined. Now, since $\overline{T}(\overline{x})\overline{x}^{3}=0$ and
$\overline{T}(\overline{x}^{4})=0$,
$\overline{T}(\overline{x}^{4})=\overline{T}(\overline{x})\overline{x}^{3}$.
This shows, using Lemma 2.2, that
$\overline{T}(\overline{xy})=\overline{T}(\overline{x})\overline{y}$.
Therefore, $T(xy)-T(x)y\in P$. Finally, by the semiprimeness of $R$, we get
the desired result.

Now we are ready to prove the first main result.

Proof of Theorem 1.3. Let $d$ be the associated $(m,n)$-Jordan derivation of
$F$. Since $R$ is a semiprime ring, $d$ is a derivation which maps $R$ into
$Z(R)$ (by Theorem 1.1). Let us denote $F-d$ by $D$. Then, we have
$(m+n)D(x^{2})=(m+n)F(x^{2})-(m+n)d(x^{2})=2mF(x)x+2nxd(x)-2md(x)x-2nxd(x)=2mD(x)x$
for all $x\in R$. Thus

$(m+n)D(x^{2})=2mD(x)x,\ x\in R.$ (2.1)

Replacing $x$ with $x^{2}$ in (2.1), we get

$(m+n)D(x^{4})=2mD(x^{2})x^{2},\ x\in R.$ (2.2)

Multiplying by $m+n$ and then using (2.1), we get

$(m+n)^{2}D(x^{4})=4m^{2}D(x)x^{3},\ x\in R.$ (2.3)

On the other hand, putting $x^{2}$ for $y$ in the relation of Lemma 2.1 and
using the fact that $D$ is a generalized $(m,n)$-Jordan derivation associated
with the zero map as an $(m,n)$-Jordan derivation, we get

$(m+n)^{2}D(x^{4})=m(n-m)D(x)x^{3}+m(3m+n)D(x)x^{3}+m(m-n)D(x^{2})x^{2},\ x\in
R.$ (2.4)

Multiplying both sides in (2.4) by $2$ we get

$2(m+n)^{2}D(x^{4})=2m(n-m)D(x)x^{3}+2m(3m+n)D(x)x^{3}+2m(m-n)D(x^{2})x^{2},\
x\in R.$ (2.5)

Combining (2.2) and (2.5), we get

$2(m+n)^{2}D(x^{4})=2m(n-m)D(x)x^{3}+2m(3m+n)D(x)x^{3}+(m+n)(m-n)D(x^{4}),\
x\in R,$ (2.6)

which gives

$(m+n)(m+3n)D(x^{4})=4m(m+n)D(x)x^{3},\ x\in R.$ (2.7)

Multiplying both sides in (2.7) by $m+n$, we get

$(m+n)^{2}(m+3n)D(x^{4})=4m(m+n)^{2}D(x)x^{3},\ x\in R.$ (2.8)

Multiplying by $m+3n$ in (2.3), we get

$(m+n)^{2}(m+3n)D(x^{4})=4m^{2}(m+3n)D(x)x^{3},\ x\in R.$ (2.9)

By comparing (2.8) and (2.9), we get

$4mn(m-n)D(x)x^{3}=0,\ x\in R.$ (2.10)

Since $R$ is a $2mn|n-m|$-torsion free ring, $D(x)x^{3}=0$ for all $x\in R$.
Applying $D(x)x^{3}=0$ in equation (2.3), we get $(m+n)^{2}D(x^{4})=0$ for all
$x\in R$. By using the torsion free restriction, we have $D(x^{4})=0$ for all
$x\in R$. Hence, $D(xy)=D(x)y$ for all $x,y\in R$ (by Lemma 2.5). Applying
this in (2.1), yields $(m+n)D(x)x=2mD(x)x$ for all $x\in R$, equivalently
$(m-n)D(x)x=0$. Since $R$ is an $|m-n|$-torsion free ring, $D(x)x=0$ for all
$x\in R$. Therefore, by Lemma 2.4, $D=0$. This completes the proof.

The second main result is proved similarly. Nevertheless, we include a proof
for completeness.

Proof of Theorem 1.6. Let $T_{0}$ be the associated $(m,n)$-Jordan centralizer
of $T$. Since $R$ is a semiprime ring, $T_{0}$ is a two-sided centralizer (by
Theorem 1.4). Let us denote $T-T_{0}$ by $D$. Then, we have
$(m+n)D(x^{2})=(m+n)T(x^{2})-(m+n)T_{0}(x^{2})=mT(x)x+nxT_{0}(x)-mT_{0}(x)x-nxT_{0}(x)=mD(x)x$
for all $x\in R.$ Thus

$(m+n)D(x^{2})=mD(x)x,\ x\in R.$ (2.11)

Replacing $x$ with $x^{2}$ in (2.11), we get

$(m+n)D(x^{4})=mD(x^{2})x^{2},\ x\in R.$ (2.12)

Multiplying by $m+n$ and then using (2.11), we get

$(m+n)^{2}D(x^{4})=m^{2}D(x)x^{3},\ x\in R.$ (2.13)

On the other hand, if we put $y=x^{2}$ in the relation of Lemma 2.3, we get

$2(m+n)^{2}D(x^{4})=mnD(x)x^{3}+m(2m+n)D(x)x^{3}-mnD(x^{2})x^{2},\ x\in R.$
(2.14)

Multiplying both sides in (2.13) by $2$ we get

$2(m+n)^{2}D(x^{4})=2m^{2}D(x)x^{3},\ x\in R.$ (2.15)

Combining (2.12) and (2.14), we get

$2(m+n)^{2}D(x^{4})=mnD(x)x^{3}+m(2m+n)D(x)x^{3}-n(m+n)D(x^{4}),\ x\in R,$
(2.16)

which implies

$(m+n)(2m+3n)D(x^{4})=2m(m+n)D(x)x^{3},\ x\in R.$ (2.17)

Multiplying both sides of above relation by $m+n$, we have

$(m+n)^{2}(2m+3n)D(x^{4})=2m(m+n)^{2}D(x)x^{3},\ x\in R.$ (2.18)

Multiplying by $(2m+3n)$ in (2.13), we get

$(m+n)^{2}(2m+3n)D(x^{4})=m^{2}(2m+3n)D(x)x^{3},\ x\in R.$ (2.19)

By comparing (2.18) and (2.19), we get

$mn(2n+m)D(x)x^{3}=0,\ x\in R.$ (2.20)

Since $R$ is a $mn(2n+m)$-torsion free ring, $D(x)x^{3}=0$ for all $x\in R$.
Applying $D(x)x^{3}=0$ in equation (2.13) and then using $(m+n)$-torsion
freeness of $R$, we get $D(x^{4})=0$ for all $x\in R$. Moreover, since $R$ is
a $2$ and a $3$-torsion free ring, by Lemma 2.5, we get $D(xy)=D(x)y$ for all
$x,y\in R$. Applying this in (2.11), yields $(m+n)D(x)x=mD(x)x$ for all $x\in
R$. So $nD(x)x=0$, which implies that $D(x)x=0$ for all $x\in R$. Therefore,
by Lemma 2.4, $D=0$. This completes the proof.

Acknowledgements. The authors would like to thank Professor Abdellah Mamouni
for useful discussions.

## References

  * [1] S. Ali and A. Fošner, On Generalized $(m,n)$-Derivations and Generalized $(m,n)$-Jordan Derivations in Rings, Algebra Colloq. 21 (2014), 411–420.
  * [2] M. Brešar, Jordan derivations on semiprime rings, Proc. Amer. Math. Soc. 104 (1988), 1003–1006.
  * [3] M. Brešar, On the distance of the composition of two derivations to the generalized derivations, Glasg. Math. J. 33 (1991), 89–93.
  * [4] M. Brešar, Introduction to noncommutative algebra, Universitext, Springer, 2014.
  * [5] M. Brešar and J. Vukman, Jordan derivations on prime rings, Bull. Aust. Math. Soc. 37 (1988), 321–322.
  * [6] D. Benkovič and D. Eremita, Characterizing left centralizers by their action on a polynomial, Publ. Math. Debercen 64 (2004), 343–351.
  * [7] J. Cusak, Jordan derivations on rings, Proc. Amer. Math. Soc. 53 (1975), 321–324.
  * [8] A. Fošner, A note on generalized $(m,n)$-Jordan centralizers, Demonstratio Math. 46 (2013), 254–262.
  * [9] I. N. Herstein, Jordan derivations of prime rings, Proc. Amer. Math. Soc. 8 (1957), 1104–1119.
  * [10] W. Jing and S. Lu, Generalized Jordan derivations on prime rings and standard operator algebras, Taiwanese J. Math. 7 (2003), 605–613.
  * [11] I. Kosi-Ulbl and J. Vukman, A note on $(m,n)$-Jordan derivation of rings and banach algebras, Bull. Aust. Math. Soc. 93 (2016), 231–237.
  * [12] I. Kosi-Ulbl and J.Vukman, On $(m,n)$-Jordan centralizers of semiprime rings, Publ. Math. Debrecen 7490 (2016), 1–9.
  * [13] T. Y. Lam, A first course in noncommutative rings, Graduate Texts in Mathematics, 123, Springer-Verlag, New York, 1991.
  * [14] J. Levitzki, Prime ideals and the lower radical, Amer. J. Math. 73 (1951), 25–29.
  * [15] J. Vukman, An identity related to centralizers in semiprime rings, Comment. Math. Univ. Carolin. 40 (1999), 447–456.
  * [16] J. Vukman, Identities with derivations and automorphisms on semiprime rings, Int. J. Math. Math. Sci. 7 (2005), 1031–1038.
  * [17] J. Vukman, A note on generalized derivations of semiprime rings, Taiwanese J. Math. 11 (2007), 367–370.
  * [18] J. Vukman, On $(m,n)$-Jordan derivations and commutativity of prime rings, Demonstratio Math. 41 (2008), 773–778.
  * [19] J. Vukman, On $(m,n)$-Jordan centralizers in rings and algebras, Glasg. Math. J. 45 (2010), 43–53.
  * [20] B. Zalar, On centralizers of semiprime rings, Comment. Math. Univ. Carolin. 32 (1991), 609–614.