application. A total of 78 studies have been chosen for this evaluation due to
their specific focus on the application of AI in edge computing. In order to
enhance our comprehension in the context of AI applied to edge computing,
these research were categorized into multiple domains, including
infrastructure, resource management, and ML model sizing, among others.
Resource supply, allocation, scheduling, and job deployment are crucial
considerations in the field of Edge AI.

## Acknowledgements

M. Golec is supported by the Ministry of Education of the Turkish Republic. B.
Ali is supported by the Ph.D. Scholarship at the Queen Mary University of
London. H. Wu is supported by the National Natural Science Foundation of China
(No. 62071327) and Tianjin Science and Technology Planning Project (No.
22ZYYYJC00020). F. Cuadrado has been supported by the HE ACES project (Grant
No. 101093126). M. Xu is supported by the National Natural Science Foundation
of China (No. 62102408), Guangdong Basic and Applied Basic Research Foundation
(No. 2024A1515010251), Shenzhen Industrial Application Projects of undertaking
the National key R & D Program of China (No. CJGJZD20210408091600002).

## References

  * [1] G. K. Walia, M. Kumar, and S. S. Gill, “Ai-empowered fog/edge resource management for iot applications: A comprehensive review, research challenges, and future perspectives,” _IEEE Communications Surveys & Tutorials_, vol. 26, no. 1, pp. 619–669, 2024.
  * [2] A. Y. Ding, E. Peltonen, T. Meuser _et al._ , “Roadmap for edge ai: A dagstuhl perspective,” pp. 28–33, 2022.
  * [3] M. Golec and S. S. Gill, “Computing: Looking back and moving forward,” _Proceedings of the 21st International Conference on Smart Business Technologies (ICSBT 2024)_ , pp. 7–14, 2024.
  * [4] S. Iftikhar, S. S. Gill, C. Song, M. Xu, M. S. Aslanpour, A. N. Toosi, J. Du, H. Wu, S. Ghosh, D. Chowdhury _et al._ , “Ai-based fog and edge computing: A systematic review, taxonomy and future directions,” _Internet of Things_ , vol. 21, p. 100674, 2023.
  * [5] Q. Duan, J. Huang, S. Hu, R. Deng, Z. Lu, and S. Yu, “Combining federated learning and edge computing toward ubiquitous intelligence in 6g network: Challenges, recent advances, and future directions,” _IEEE Communications Surveys & Tutorials_, 2023.
  * [6] R. Singh and S. S. Gill, “Edge ai: a survey,” _Internet of Things and Cyber-Physical Systems_ , vol. 3, pp. 71–92, 2023.
  * [7] Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. B. Letaief, “Communication-efficient edge ai: Algorithms and systems,” _IEEE Communications Surveys & Tutorials_, vol. 22, no. 4, pp. 2167–2191, 2020.
  * [8] D. Liu, H. Kong, X. Luo, W. Liu, and R. Subramaniam, “Bringing ai to edge: From deep learning’s perspective,” _Neurocomputing_ , vol. 485, pp. 297–320, 2022.
  * [9] A. Rocha, M. Monteiro _et al._ , “Edge ai for internet of medical things: A literature review,” _Computers and Electrical Engineering_ , vol. 116, p. 109202, 2024.
  * [10] W. Su, L. Li, F. Liu, M. He, and X. Liang, “Ai on the edge: a comprehensive review,” _Artificial Intelligence Review_ , vol. 55, no. 8, pp. 6125–6183, 2022.
  * [11] W. Zhang, S. Zeadally, W. Li, H. Zhang, J. Hou, and V. C. M. Leung, “Edge ai as a service: Configurable model deployment and delay-energy optimization with result quality constraints,” _IEEE Transactions on Cloud Computing_ , vol. 11, no. 2, pp. 1954–1969, 2023.
  * [12] H. N. Qureshi, U. Masood, M. Manalastas, S. M. A. Zaidi, H. Farooq, J. Forgeat, M. Bouton, S. Bothe, P. Karlsson, A. Rizwan _et al._ , “Towards addressing training data scarcity challenge in emerging radio access networks: A survey and framework,” _IEEE Communications Surveys & Tutorials_, 2023.
  * [13] M. Golec, S. Iftikhar, P. Prabhakaran, S. S. Gill, and S. Uhlig, “Qos analysis for serverless computing using machine learning,” in _Serverless Computing: Principles and Paradigms_. Springer, 2023, pp. 175–192.
  * [14] S. Shahriar, S. Allana, S. M. Hazratifard, and R. Dara, “A survey of privacy risks and mitigation strategies in the artificial intelligence life cycle,” _IEEE Access_ , vol. 11, pp. 61 829–61 854, 2023.
  * [15] M. Kumar, G. K. Walia, H. Shingare, S. Singh, and S. S. Gill, “Ai-based sustainable and intelligent offloading framework for iiot in collaborative cloud-fog environments,” _IEEE Transactions on Consumer Electronics_ , 2023.
  * [16] K. Hoffpauir, J. Simmons, N. Schmidt, R. Pittala, I. Briggs, S. Makani, and Y. Jararweh, “A survey on edge intelligence and lightweight machine learning support for future applications and services,” _ACM Journal of Data and Information Quality_ , vol. 15, no. 2, pp. 1–30, 2023.
  * [17] S. S. Gill and R. Buyya, “Transforming research with quantum computing,” _Journal of Economy and Technology_ , vol. 2, pp. 1–11, 2024.
  * [18] N. Huang, C. Dou, Y. Wu, L. Qian, and R. Lu, “Energy-efficient integrated sensing and communication: A multi-access edge computing design,” _IEEE Wireless Communications Letters_ , 2023.
  * [19] P. Verma, S. K. Sood, H. Kaur, M. Kumar, H. Wu, and S. S. Gill, “Data driven stochastic game network-based smart home monitoring system using iot-enabled edge computing environments,” _IEEE Transactions on Consumer Electronics_ , 2024.
  * [20] S. S. Gill, H. Wu, P. Patros, C. Ottaviani, P. Arora, V. C. Pujol, D. Haunschild, A. K. Parlikad, O. Cetinkaya, H. Lutfiyya _et al._ , “Modern computing: Vision and challenges,” _Telematics and Informatics Reports_ , p. 100116, 2024.
  * [21] S. Velu, S. S. Gill, S. S. Murugesan, H. Wu, and X. Li, “Cloudaibus: A testbed for ai based cloud computing environments,” _Cluster Computing_ , 2024. [Online]. Available: https://doi.org/10.1007/s10586-024-04562-9
  * [22] X. Xu, Q. Huang, X. Yin, M. Abbasi, M. R. Khosravi, and L. Qi, “Intelligent offloading for collaborative smart city services in edge computing,” _IEEE Internet of Things Journal_ , vol. 7, no. 9, pp. 7919–7927, 2020.
  * [23] H. Huang, K. Peng, and X. Xu, “Collaborative computation offloading for smart cities in mobile edge computing,” in _2020 IEEE 13th International conference on cloud computing (CLOUD)_. IEEE, 2020, pp. 176–183.
  * [24] C. Li, H. Wang, and R. Song, “Intelligent offloading for noma-assisted mec via dual connectivity,” _IEEE Internet of Things Journal_ , vol. 8, no. 4, pp. 2802–2813, 2020.
  * [25] Y. Zhang, X. Liu, J. Xu, D. Yuan, and X. Li, “A novel adaptive computation offloading strategy for collaborative dnn inference over edge devices,” in _2022 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)_. IEEE, 2022, pp. 378–385.
  * [26] A. Fresa and J. P. V. Champati, “An offloading algorithm for maximizing inference accuracy on edge device in an edge intelligence system,” in _Proceedings of the 25th International ACM Conference on Modeling Analysis and Simulation of Wireless and Mobile Systems_ , 2022, pp. 15–23.
  * [27] I. Khan, S. Raza, W. u. Rehman, R. Khan, K. Nahida, and X. Tao, “A deep learning-based algorithm for energy and performance optimization of computational offloading in mobile edge computing,” _Wireless Communications and Mobile Computing_ , vol. 2023, no. 1, p. 1357343, 2023.
  * [28] M. Du, Y. Wang, K. Ye, and C. Xu, “Algorithmics of cost-driven computation offloading in the edge-cloud environment,” _IEEE Transactions on Computers_ , vol. 69, no. 10, pp. 1519–1532, 2020.
  * [29] A. S. Choudhury, T. Halder, A. Basak, and D. Chakravarty, “Implementation of artificial intelligence (ai) in smart manufacturing: A status review,” in _International Conference on Computational Intelligence and Smart Communication_. Springer, 2022, pp. 73–85.
  * [30] S. J. Plathottam, A. Rzonca, R. Lakhnori, and C. O. Iloeje, “A review of artificial intelligence applications in manufacturing operations,” _Journal of Advanced Manufacturing and Processing_ , vol. 5, no. 3, p. e10159, 2023.
  * [31] C. Yang, S. Lan, L. Wang, W. Shen, and G. G. Huang, “Big data driven edge-cloud collaboration architecture for cloud manufacturing: a software defined perspective,” _IEEE access_ , vol. 8, pp. 45 938–45 950, 2020.
  * [32] J. Moon and J. Jeong, “Smart manufacturing scheduling system: Dqn based on cooperative edge computing,” in _2021 15th international conference on ubiquitous information management and communication (IMCOM)_. IEEE, 2021, pp. 1–8.
  * [33] I. Mishra, S. Kumar, and N. Gupta, “Smart manufacturing: Opportunities and challenges overcome by industry 4.0,” _Soft Computing in Materials Development and its Sustainability in the Manufacturing Sector_ , pp. 179–182, 2022.
  * [34] Y. Zhang, D. Tang, H. Zhu, S. Zhou, and Z. Zhao, “An efficient iiot gateway for cloud–edge collaboration in cloud manufacturing,” _Machines_ , vol. 10, no. 10, p. 850, 2022.
  * [35] P. Mohanram, A. Gilerson, R. Schmitt _et al._ , “Architecture for edge-based predictive maintenance of machines using federated learning and multi sensor platforms,” 2023.
  * [36] H. Rizk, A. Chaibet, and A. Kribèche, “Model-based control and model-free control techniques for autonomous vehicles: A technical survey,” _Applied Sciences_ , vol. 13, no. 11, p. 6700, 2023.
  * [37] Z. Ning, H. Hu, X. Wang, L. Guo, S. Guo, G. Wang, and X. Gao, “Mobile edge computing and machine learning in the internet of unmanned aerial vehicles: a survey,” _ACM Computing Surveys_ , vol. 56, no. 1, pp. 1–31, 2023.
  * [38] M. Ahmed, M. A. Mirza, S. Raza, H. Ahmad, F. Xu, W. U. Khan, Q. Lin, and Z. Han, “Vehicular communication network enabled cav data offloading: A review,” _IEEE Transactions on Intelligent Transportation Systems_ , 2023.
  * [39] D. Xue, Y. Guo, N. Li, X. Song, and L. Zhang, “Cross-domain coordination of resource allocation and route planning for the edge computing-enabled multi-connected vehicles,” _Journal of Cloud computing_ , vol. 12, no. 1, p. 33, 2023.
  * [40] G. Ming, “Exploration of the intelligent control system of autonomous vehicles based on edge computing,” _PLoS One_ , vol. 18, no. 2, p. e0281294, 2023.
  * [41] M. Firdaus and K.-H. Rhee, “A joint framework to privacy-preserving edge intelligence in vehicular networks,” in _International Conference on Information Security Applications_. Springer, 2022, pp. 156–167.
  * [42] B. Atan, M. Basaran, N. Calik, S. T. Basaran, G. Akkuzu, and L. Durak-Ata, “Ai-empowered fast task execution decision for delay-sensitive iot applications in edge computing networks,” _IEEE Access_ , vol. 11, pp. 1324–1334, 2022.
  * [43] T. Anees, Q. Habib, A. S. Al-Shamayleh, W. Khalil, M. A. Obaidat, and A. Akhunzada, “The integration of wot and edge computing: Issues and challenges,” _Sustainability_ , vol. 15, no. 7, p. 5983, 2023.
  * [44] O. Ajibuwa, B. Hamdaoui, and A. A. Yavuz, “A survey on ai/ml-driven intrusion and misbehavior detection in networked autonomous systems: Techniques, challenges and opportunities,” _arXiv preprint arXiv:2305.05040_ , 2023.
  * [45] P. Verma and S. K. Sood, “Fog assisted-iot enabled patient health monitoring in smart homes,” _IEEE Internet of Things Journal_ , vol. 5, no. 3, pp. 1789–1796, 2018.
  * [46] T. A. Shaikh, T. Rasool, and P. Verma, “Machine intelligence and medical cyber-physical system architectures for smart healthcare: Taxonomy, challenges, opportunities, and possible solutions,” _Artificial Intelligence in Medicine_ , p. 102692, 2023.
  * [47] S. T. Ahmed, S. M. Basha, M. Ramachandran, M. Daneshmand, and A. H. Gandomi, “An edge-ai enabled autonomous connected ambulance route resource recommendation protocol (aca-r3) for ehealth in smart cities,” _IEEE Internet of Things Journal_ , 2023.
  * [48] S. Misra, S. Pal, P. K. Deb, and E. Gupta, “Kedge: Fuzzy-based multi-ai model coalescence solution for mobile healthcare system,” _IEEE Systems Journal_ , 2023.
  * [49] C. Chakraborty, S. M. Nagarajan, G. G. Devarajan, T. Ramana, and R. Mohanty, “Intelligent ai-based healthcare cyber security system using multi-source transfer learning method,” _ACM Transactions on Sensor Networks_ , 2023.
  * [50] K. Dvijotham, J. Winkens, M. Barsbey, S. Ghaisas, R. Stanforth, N. Pawlowski, P. Strachan, Z. Ahmed, S. Azizi, Y. Bachrach _et al._ , “Enhancing the reliability and accuracy of ai-enabled diagnosis via complementarity-driven deferral to clinicians,” _Nature Medicine_ , vol. 29, no. 7, pp. 1814–1820, 2023.
  * [51] M. Golec, S. S. Gill, R. Bahsoon, and O. Rana, “Biosec: A biometric authentication framework for secure and private communication among edge devices in iot and industry 4.0,” _IEEE Consumer Electronics Magazine_ , vol. 11, no. 2, pp. 51–56, 2020.
  * [52] M. Golec, S. S. Gill, F. Cuadrado, A. K. Parlikad, M. Xu, H. Wu, and S. Uhlig, “Atom: Ai-powered sustainable resource management for serverless edge computing environments,” _IEEE Transactions on Sustainable Computing_ , 2023.
  * [53] M. Golec, R. Ozturac, Z. Pooranian, S. S. Gill, and R. Buyya, “Ifaasbus: A security-and privacy-based lightweight framework for serverless computing using iot and machine learning,” _IEEE Transactions on Industrial Informatics_ , vol. 18, no. 5, pp. 3522–3529, 2021.
  * [54] M. Golec, S. S. Gill, A. K. Parlikad, and S. Uhlig, “Healthfaas: Ai based smart healthcare system for heart patients using serverless computing,” _IEEE Internet of Things Journal_ , 2023.
  * [55] N. Peter, “Fog computing and its real time applications,” _Int. J. Emerg. Technol. Adv. Eng_ , vol. 5, no. 6, pp. 266–269, 2015.
  * [56] S. Iftikhar, M. Golec, D. Chowdhury, S. S. Gill, and S. Uhlig, “Fog computing based router-distributor application for sustainable smart home,” in _2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring)_. IEEE, 2022, pp. 1–5.
  * [57] M. Golec, M. Golec, M. Xu, H. Wu, S. S. Gill, and S. Uhlig, “Priceless: Privacy enhanced ai-driven scalable framework for iot applications in serverless edge computing environments,” _Internet Technology Letters_ , p. e510, 2024.
  * [58] M. Golec, S. S. Gill, H. Wu, T. C. Can, M. Golec, O. Cetinkaya, F. Cuadrado, A. K. Parlikad, and S. Uhlig, “Master: Machine learning-based cold start latency prediction framework in serverless edge computing environments for industry 4.0,” _IEEE Journal of Selected Areas in Sensors_ , 2024.
  * [59] S. S. Gill, “A manifesto for modern fog and edge computing: Vision, new paradigms, opportunities, and future directions,” in _Operationalizing Multi-Cloud Environments: Technologies, Tools and Use Cases_. Springer, 2021, pp. 237–253.
  * [60] A. R. Nandhakumar, A. Baranwal, P. Choudhary, M. Golec, and S. S. Gill, “Edgeaisim: A toolkit for simulation and modelling of ai models in edge computing environments,” _Measurement: Sensors_ , vol. 31, p. 100939, 2024.
  * [61] C. P. Lee, F. T. J. Leng, R. A. A. Habeeb, M. A. Amanullah, and M. H. ur Rehman, “Edge computing-enabled secure and energy-efficient smart parking: A review,” _Microprocessors and Microsystems_ , vol. 93, p. 104612, 2022.
  * [62] D. R. Patrikar and M. R. Parate, “Anomaly detection using edge computing in video surveillance system,” _International Journal of Multimedia Information Retrieval_ , vol. 11, no. 2, pp. 85–110, 2022.
  * [63] L. Wu, R. Zhang, R. Zhou, and D. Wu, “An edge computing based data detection scheme for traffic light at intersections,” _Computer Communications_ , vol. 176, pp. 91–98, 2021.
  * [64] Y. Liu, “Open university chinese language and literature teaching model based on nlp technology and mobile edge computing,” _Mobile Information Systems_ , vol. 2022, no. 1, p. 4149492, 2022.
  * [65] M. Golec, S. S. Gill, M. Golec, M. Xu, S. K. Ghosh, S. S. Kanhere, O. Rana, and S. Uhlig, “Blockfaas: Blockchain-enabled serverless computing framework for ai-driven iot healthcare applications,” _Journal of Grid Computing_ , vol. 21, no. 4, p. 63, 2023.
  * [66] K. Cao, Y. Liu, G. Meng, and Q. Sun, “An overview on edge computing research,” _IEEE access_ , vol. 8, pp. 85 714–85 728, 2020.
  * [67] M. Satyanarayanan, “The emergence of edge computing,” _Computer_ , vol. 50, no. 1, pp. 30–39, 2017.
  * [68] S. Keele _et al._ , “Guidelines for performing systematic literature reviews in software engineering,” Citeseer, Tech. Rep., 2007.
  * [69] B. A. Kitchenham, “Systematic review in software engineering: where we are and where we should be going,” in _Proceedings of the 2nd international workshop on Evidential assessment of software technologies_ , 2012, pp. 1–2.
  * [70] P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, “Lessons from applying the systematic literature review process within the software engineering domain,” _Journal of systems and software_ , vol. 80, no. 4, pp. 571–583, 2007.
  * [71] G. M. Tawfik, K. A. S. Dila, M. Y. F. Mohamed, D. N. H. Tam, N. D. Kien, A. M. Ahmed, and N. T. Huy, “A step by step guide for conducting a systematic review and meta-analysis with simulation data,” _Tropical medicine and health_ , vol. 47, pp. 1–9, 2019.
  * [72] S. P. Singh, A. Sharma, and R. Kumar, “Design and exploration of load balancers for fog computing using fuzzy logic,” _Simulation Modelling Practice and Theory_ , vol. 101, p. 102017, 2020.
  * [73] K. Gos and W. Zabierowski, “The comparison of microservice and monolithic architecture,” in _2020 IEEE XVIth International Conference on the Perspective Technologies and Methods in MEMS Design (MEMSTECH)_. IEEE, 2020, pp. 150–153.
  * [74] B. Errasti-Alcala and R. Fernandez-Recio, “Meta-heuristic approach for single-snapshot 2d-doa and frequency estimation: Array topologies and performance analysis [wireless corner],” _IEEE Antennas and Propagation Magazine_ , vol. 55, no. 1, pp. 222–238, 2013.
  * [75] Y. Himeur, A. Sayed, A. Alsalemi, F. Bensaali, and A. Amira, “Edge ai for internet of energy: Challenges and perspectives,” _ArXiv_ , vol. abs/2311.16851, 2023.
  * [76] S. S. Gill, “Quantum and blockchain based serverless edge computing: A vision, model, new trends and future directions,” _Internet Technology Letters_ , vol. 7, no. 1, p. e275, 2024.
  * [77] S. S. Gill, M. Xu, C. Ottaviani, P. Patros, R. Bahsoon, A. Shaghaghi, M. Golec, V. Stankovski, H. Wu, A. Abraham _et al._ , “Ai for next generation computing: Emerging trends and future directions,” _Internet of Things_ , vol. 19, p. 100514, 2022.
  * [78] Z. Sharif, L. T. Jung, M. Ayaz, M. Yahya, and S. Pitafi, “Priority-based task scheduling and resource allocation in edge computing for health monitoring system,” _Journal of King Saud University-Computer and Information Sciences_ , vol. 35, no. 2, pp. 544–559, 2023.
  * [79] Z. Zhuang, Y. Li, Y. Sun, W. Qin, and Z.-H. Sun, “Network-based dynamic dispatching rule generation mechanism for real-time production scheduling problems with dynamic job arrivals,” _Robotics and Computer-Integrated Manufacturing_ , vol. 73, p. 102261, 2022.
  * [80] H. Singh, S. Tyagi, P. Kumar, S. S. Gill, and R. Buyya, “Metaheuristics for scheduling of heterogeneous tasks in cloud computing environments: Analysis, performance evaluation, and future directions,” _Simulation Modelling Practice and Theory_ , vol. 111, p. 102353, 2021.
  * [81] F. Desai, D. Chowdhury, R. Kaur, M. Peeters, R. C. Arya, G. S. Wander, S. S. Gill, and R. Buyya, “Healthcloud: A system for monitoring health status of heart patients using machine learning and cloud computing,” _Internet of Things_ , vol. 17, p. 100485, 2022.
  * [82] S. Sheng, P. Chen, Z. Chen, L. Wu, and Y. Yao, “Deep reinforcement learning-based task scheduling in iot edge computing,” _Sensors_ , vol. 21, no. 5, p. 1666, 2021.
  * [83] B. R. Kiran, I. Sobh, V. Talpaert, P. Mannion, A. A. Al Sallab, S. Yogamani, and P. Pérez, “Deep reinforcement learning for autonomous driving: A survey,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 23, no. 6, pp. 4909–4926, 2021.
  * [84] Z. Zhong, M. Xu, M. A. Rodriguez, C. Xu, and R. Buyya, “Machine learning-based orchestration of containers: A taxonomy and future directions,” _ACM Computing Surveys (CSUR)_ , vol. 54, no. 10s, pp. 1–35, 2022.
  * [85] S. Iftikhar, U. Raj, S. Tuli, M. Golec, D. Chowdhury, S. S. Gill, and S. Uhlig, “Tesco: Multiple simulations based ai-augmented fog computing for qos optimization,” in _2022 IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld/UIC/ScalCom/DigitalTwin/PriComp/Meta)_. IEEE, 2022, pp. 2092–2099.
  * [86] Z. M. Nayeri, T. Ghafarian, and B. Javadi, “Application placement in fog computing with ai approach: Taxonomy and a state of the art survey,” _Journal of Network and Computer Applications_ , vol. 185, p. 103078, 2021.
  * [87] O. Carvalho, M. Garcia, E. Roloff, E. D. Carreño, and P. O. Navaux, “Iot workload distribution impact between edge and cloud computing in a smart grid application,” in _High Performance Computing: 4th Latin American Conference, CARLA 2017, Buenos Aires, Argentina, and Colonia del Sacramento, Uruguay, September 20-22, 2017, Revised Selected Papers 4_. Springer, 2018, pp. 203–217.
  * [88] C. Nguyen, C. Klein, and E. Elmroth, “Multivariate lstm-based location-aware workload prediction for edge data centers,” in _2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)_. IEEE, 2019, pp. 341–350.
  * [89] P. R. Winters, “Forecasting sales by exponentially weighted moving averages,” _Management science_ , vol. 6, no. 3, pp. 324–342, 1960.
  * [90] Y. Qiao, M. S. Munir, A. Adhikary, A. D. Raha, S. H. Hong, and C. S. Hong, “A framework for multi-prototype based federated learning: Towards the edge intelligence,” in _2023 International Conference on Information Networking (ICOIN)_. IEEE, 2023, pp. 134–139.
  * [91] A. Briouya, H. Briouya, and A. Choukri, “Overview of the progression of state-of-the-art language models,” _TELKOMNIKA (Telecommunication Computing Electronics and Control)_ , vol. 22, no. 4, pp. 897–909, 2024.
  * [92] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.
  * [93] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in _International conference on machine learning_. PMLR, 2019, pp. 6105–6114.
  * [94] ——, “Efficientnetv2: Smaller models and faster training,” in _International conference on machine learning_. PMLR, 2021, pp. 10 096–10 106.
  * [95] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” _arXiv preprint arXiv:1704.04861_ , 2017.
  * [96] X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 6848–6856.
  * [97] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [98] Y. Xiong, Y. Sun, L. Xing, and Y. Huang, “Extend cloud to edge with kubeedge,” in _2018 IEEE/ACM Symposium On Edge Computing (SEC)_. IEEE, 2018, pp. 373–377.
  * [99] P. Singh, A. Kaur, and S. S. Gill, “Machine learning for cloud, fog, edge and serverless computing environments: Comparisons, performance evaluation benchmark and future directions,” _International Journal of Grid and Utility Computing_ , vol. 13, no. 4, pp. 447–457, 2022.
  * [100] Y. You, Z. Zhang, C.-J. Hsieh, J. Demmel, and K. Keutzer, “Fast deep neural network training on distributed systems and cloud tpus,” _IEEE Transactions on Parallel and Distributed Systems_ , vol. 30, no. 11, pp. 2449–2462, 2019.
  * [101] Y. Sun and A. M. Kist, “Deep learning on edge tpus,” _arXiv preprint arXiv:2108.13732_ , 2021.
  * [102] E. Casalicchio and S. Iannucci, “The state-of-the-art in container technologies: Application, orchestration and security,” _Concurrency and Computation: Practice and Experience_ , vol. 32, no. 17, p. e5668, 2020.
  * [103] J. Zhang, C. Lu, G. Cheng, T. Guo, J. Kang, X. Zhang, X. Yuan, and X. Yan, “A blockchain-based trusted edge platform in edge computing environment,” _Sensors_ , vol. 21, no. 6, p. 2126, 2021.
  * [104] T. Wang, G. Zhang, A. Liu, M. Z. A. Bhuiyan, and Q. Jin, “A secure iot service architecture with an efficient balance dynamics based on cloud and edge computing,” _IEEE Internet of Things Journal_ , vol. 6, no. 3, pp. 4831–4843, 2018.
  * [105] A. Gharaibeh, M. A. Salahuddin, S. J. Hussini, A. Khreishah, I. Khalil, M. Guizani, and A. Al-Fuqaha, “Smart cities: A survey on data management, security, and enabling technologies,” _IEEE Communications Surveys & Tutorials_, vol. 19, no. 4, pp. 2456–2501, 2017.
  * [106] B. Krebs, “Krebsonsecurity hit with record ddos,” 2016, accessed: 2024-06-15. [Online]. Available: https://krebsonsecurity.com/2016/09/krebsonsecurity-hit-with-recordddos/
  * [107] Dyn, “Dyn analysis summary of friday october 21 attack,” 2016, accessed: 2024-06-15. [Online]. Available: http://dyn.com/blog/dyn-analysis-summary-of-friday-october-21attack/
  * [108] K. Bhardwaj, J. C. Miranda, and A. Gavrilovska, “Towards IoT-DDoS prevention using edge computing,” in _USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18)_. Boston, MA: USENIX Association, Jul. 2018. [Online]. Available: https://www.usenix.org/conference/hotedge18/presentation/bhardwaj
  * [109] O. Oleghe, “Container placement and migration in edge computing: Concept and scheduling models,” _IEEE Access_ , vol. 9, pp. 68 028–68 043, 2021.
  * [110] Z. Li, Z. Yang, S. Xie, W. Chen, and K. Liu, “Credit-based payments for fast computing resource trading in edge-assisted internet of things,” _IEEE Internet of Things Journal_ , vol. 6, no. 4, pp. 6606–6617, 2019.
  * [111] X. Zhang, Y. Zhong, P. Liu, F. Zhou, and Y. Wang, “Resource allocation for a uav-enabled mobile-edge computing system: Computation efficiency maximization,” _IEEE Access_ , vol. 7, pp. 113 345–113 354, 2019.
  * [112] T. X. Tran and D. Pompili, “Joint task offloading and resource allocation for multi-server mobile-edge computing networks,” _IEEE Transactions on Vehicular Technology_ , vol. 68, no. 1, pp. 856–868, 2018.
  * [113] Y. Wei, L. Pan, S. Liu, L. Wu, and X. Meng, “Drl-scheduling: An intelligent qos-aware job scheduling framework for applications in clouds,” _IEEE Access_ , vol. 6, pp. 55 112–55 125, 2018.
  * [114] C. Carrión, “Kubernetes scheduling: Taxonomy, ongoing issues and challenges,” _ACM Computing Surveys_ , vol. 55, no. 7, pp. 1–37, 2022.
  * [115] P. S. Junior, D. Miorandi, and G. Pierre, “Stateful container migration in geo-distributed environments,” in _2020 IEEE international conference on cloud computing technology and science (CloudCom)_. IEEE, 2020, pp. 49–56.
  * [116] S. S. Murugesan, S. Velu, M. Golec, H. Wu, and S. S. Gill, “Neural networks based smart e-health application for the prediction of tuberculosis using serverless computing,” _IEEE Journal of Biomedical and Health Informatics_ , pp. 1–12, 2024.
  * [117] R. Amin, S. Vadlamudi, and M. M. Rahaman, “Opportunities and challenges of data migration in cloud,” _Engineering International_ , vol. 9, no. 1, pp. 41–50, 2021.
  * [118] M. Chiang and T. Zhang, “Fog and iot: An overview of research opportunities,” _IEEE Internet of things journal_ , vol. 3, no. 6, pp. 854–864, 2016.
  * [119] S. Iftikhar, M. Golec, D. Chowdhury, S. S. Gill, and S. Uhlig, “Fogdlearner: A deep learning-based cardiac health diagnosis framework using fog computing,” in _Proceedings of the 2022 Australasian Computer Science Week_ , 2022, pp. 136–144.
  * [120] M. Imdoukh, I. Ahmad, and M. G. Alfailakawi, “Machine learning-based auto-scaling for containerized applications,” _Neural Computing and Applications_ , vol. 32, no. 13, pp. 9745–9760, 2020.
  * [121] The Prometheus Authors, “Prometheus - monitoring system & time series database,” https://prometheus.io/, 2024, accessed: 2024-06-20.
  * [122] F. Klinaku, M. Frank, and S. Becker, “Caus: An elasticity controller for a containerized microservice,” in _Companion of the 2018 ACM/SPEC International Conference on Performance Engineering_ , 2018, pp. 93–98.
  * [123] A. H. Ali, “A survey on vertical and horizontal scaling platforms for big data analytics,” _International Journal of Integrated Engineering_ , vol. 11, no. 6, pp. 138–150, 2019.
  * [124] T.-T. Nguyen, Y.-J. Yeom, T. Kim, D.-H. Park, and S. Kim, “Horizontal pod autoscaling in kubernetes for elastic container orchestration,” _Sensors_ , vol. 20, no. 16, p. 4621, 2020.
  * [125] F. Rossi, M. Nardelli, and V. Cardellini, “Horizontal and vertical scaling of container-based applications using reinforcement learning,” in _2019 IEEE 12th International Conference on Cloud Computing (CLOUD)_. IEEE, 2019, pp. 329–338.
  * [126] H. Hu and C. Jiang, “Edge intelligence: Challenges and opportunities,” in _2020 International Conference on Computer, Information and Telecommunication Systems (CITS)_ , 2020, pp. 1–5.
  * [127] C. Dai and Q. Song, “Heuristic computing methods for contact plan design in the spatial-node-based internet of everything,” _China Communications_ , vol. 16, no. 3, pp. 53–68, 2019.
  * [128] C. Mattmann, 2020.
  * [129] M. Kaloev and G. Krastev, “Experiments focused on exploration in deep reinforcement learning,” in _2021 5th International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT)_ , 2021, pp. 351–355.
  * [130] C.-H. Hong and B. Varghese, “Resource management in fog/edge computing: a survey on architectures, infrastructure, and algorithms,” _ACM Computing Surveys (CSUR)_ , vol. 52, no. 5, pp. 1–37, 2019.
  * [131] M. Ebrahimi, A. d. S. Veith, M. Gabel, and E. de Lara, “Combining dnn partitioning and early exit,” in _Proceedings of the 5th International Workshop on Edge Systems, Analytics and Networking_ , 2022, pp. 25–30.
  * [132] Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” _ACM SIGARCH Computer Architecture News_ , vol. 45, no. 1, pp. 615–629, 2017.
  * [133] C. Liang, S. Zuo, M. Chen, H. Jiang, X. Liu, P. He, T. Zhao, and W. Chen, “Super tickets in pre-trained language models: From model compression to improving generalization,” _arXiv preprint arXiv:2105.12002_ , 2021.
  * [134] Z. Sanaei, S. Abolfazli, A. Gani, and R. Buyya, “Heterogeneity in mobile cloud computing: taxonomy and open challenges,” _IEEE Communications Surveys & Tutorials_, vol. 16, no. 1, pp. 369–392, 2013.
  * [135] G. Marin, “Network security basics,” _IEEE Security & Privacy_, vol. 3, no. 6, pp. 68–72, 2005.
  * [136] C. Puliafito, C. Vallati, E. Mingozzi, G. Merlino, F. Longo, and A. Puliafito, “Container migration in the fog: A performance evaluation,” _Sensors_ , vol. 19, no. 7, p. 1488, 2019.
  * [137] G. Singh and P. Singh, “A taxonomy and survey on container migration techniques in cloud computing,” _Sustainable Development Through Engineering Innovations: Select Proceedings of SDEI 2020_ , pp. 419–429, 2021.
  * [138] S. Saha, K. Banerjee, S. Ghosh, S. Mitra, and D. Pal, “Ai-driven edge computing for iot: A comprehensive survey and future directions,” _International Journal of Advanced Research in Science, Communication and Technology_ , 2023.
  * [139] S. Ghafouri, A. Karami, D. B. Bakhtiarvan, A. S. Bigdeli, S. S. Gill, and J. Doyle, “Mobile-kube: Mobility-aware and energy-efficient service orchestration on kubernetes edge servers,” in _2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC)_. IEEE, 2022, pp. 82–91.
  * [140] Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. Letaief, “Communication-efficient edge ai: Algorithms and systems,” _IEEE Communications Surveys & Tutorials_, vol. 22, pp. 2167–2191, 2020.
  * [141] J. Du, M. Xu, S. S. Gill, and H. Wu, “Computation energy efficiency maximization for intelligent reflective surface-aided wireless powered mobile edge computing,” _IEEE Transactions on Sustainable Computing_ , 2023.
  * [142] L. A. Yousef, H. Yousef, and L. Rocha‐Meneses, “Artificial intelligence for management of variable renewable energy systems: A review of current status and future directions,” _Energies_ , 2023.
  * [143] D. Chowdhury, A. Das, A. Dey, S. Banerjee, M. Golec, D. Kollias, M. Kumar, G. Kaur, R. Kaur, R. C. Arya _et al._ , “Covidetector: A transfer learning-based semi supervised approach to detect covid-19 using cxr images,” _BenchCouncil Transactions on Benchmarks, Standards and Evaluations_ , vol. 3, no. 2, p. 100119, 2023.
  * [144] S. Zhu, K. Ota, and M. Dong, “Energy-efficient artificial intelligence of things with intelligent edge,” _IEEE Internet of Things Journal_ , vol. 9, no. 10, pp. 7525–7532, 2022.
  * [145] J. Doyle, M. Golec, and S. S. Gill, “Blockchainbus: A lightweight framework for secure virtual machine migration in cloud federations using blockchain,” _Security and Privacy_ , vol. 5, no. 2, p. e197, 2022.
  * [146] M. Golec, D. Chowdhury, S. Jaglan, S. S. Gill, and S. Uhlig, “Aiblock: Blockchain based lightweight framework for serverless computing using ai,” in _2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)_. IEEE, 2022, pp. 886–892.
  * [147] R. Singh and S. S. Gill, “Next generation edge computing: A roadmap to net zero emissions,” _Journal of Economy and Technology_ , vol. 1, pp. 208–221, 2023.
  * [148] I. Kovalenko, K. Barton, J. Moyne, and D. M. Tilbury, “Opportunities and challenges to integrate artificial intelligence into manufacturing systems: Thoughts from a panel discussion [opinion],” _IEEE Robotics & Automation Magazine_, vol. 30, no. 2, pp. 109–112, 2023.
  * [149] G. Nain, K. Pattanaik, and G. Sharma, “Towards edge computing in intelligent manufacturing: Past, present and future,” _Journal of Manufacturing Systems_ , vol. 62, pp. 588–611, 2022.
  * [150] L. N. Thalluri, S. N. Venkat, C. V. V. D. Prasad, D. V. Kumar, K. P. Kumar, A. V. N. Sarma, and S. D. Adapa, “Artificial intelligence enabled smart city iot system using edge computing,” in _2021 2nd International Conference on Smart Electronics and Communication (ICOSEC)_. IEEE, 2021, pp. 12–20.
  * [151] X. Guo and X. Hong, “Dqn for smart transportation supporting v2v mobile edge computing,” in _2023 IEEE International Conference on Smart Computing (SMARTCOMP)_. IEEE, 2023, pp. 204–206.
  * [152] M. Golec, G. K. Walia, M. Kumar, F. Cuadrado, S. S. Gill, and S. Uhlig, “Cold start latency in serverless computing: A systematic review, taxonomy, and future directions,” _arXiv preprint arXiv:2310.08437_ , 2023.
  * [153] K. Senjab, S. Abbas, N. Ahmed, and A. u. R. Khan, “A survey of kubernetes scheduling algorithms,” _Journal of Cloud Computing_ , vol. 12, no. 1, p. 87, 2023.
  * [154] M. Golec, S. S. Gill, H. Wu, T. C. Can, M. Golec, O. Cetinkaya, F. Cuadrado, A. K. Parlikad, and S. Uhlig, “Master: Machine learning-based cold start latency prediction framework in serverless edge computing environments for industry 4.0,” _IEEE Journal of Selected Areas in Sensors_ , vol. 1, pp. 36–48, 2024.
  * [155] S. S. Gill _et al._ , “Quantum computing: Vision and challenges,” _Technical Report, arXiv preprint arXiv:2403.02240_ , pp. 1–11, 2024.
  * [156] M. Golec, E. S. Hatay, M. Golec _et al._ , “Quantum cloud computing: Trends and challenges,” _Journal of Economy and Technology_ , vol. 1, pp. 1–11, 2024.
  * [157] S. J. Nawaz, S. K. Sharma, S. Wyne, M. N. Patwary, and M. Asaduzzaman, “Quantum machine learning for 6g communication networks: State-of-the-art and vision for the future,” _IEEE Access_ , vol. 7, pp. 46 317–46 350, 2019.
  * [158] A. Hummelholm, “Ai-based quantum-safe cybersecurity automation and orchestration for edge intelligence in future networks,” _European Conference on Cyber Warfare and Security_ , 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:259453663
  * [159] J. K. Samriya, M. Kumar, and S. S. Gill, “Secured data offloading using reinforcement learning and markov decision process in mobile edge computing,” _International Journal of Network Management_ , vol. 33, no. 5, p. e2243, 2023.
  * [160] J. Yang, T. Baker, S. S. Gill, X. Yang, W. Han, and Y. Li, “A federated learning attack method based on edge collaboration via cloud,” _Software: Practice and Experience_.
  * [161] E. Badidi, “Edge ai for early detection of chronic diseases and the spread of infectious diseases: Opportunities, challenges, and future directions,” _Future Internet_ , vol. 15, no. 11, 2023. [Online]. Available: https://www.mdpi.com/1999-5903/15/11/370


subsection hansen customcustom0custom.5ptchaptercustom chapterplain chapter0
chapter

hansen

Doctoral Thesis

Aligning language models
with human preferences

A thesis submitted in fulfilment of the requirements
for the degree of Doctor of Philosophy
in the
Department of Informatics
Faculty of Engineering and Informatics

_Author:_

Tomasz Korbak

_Supervisors:_

Christopher L. Buckley

Anil Seth

18 September 2023

Acknowledgements

I am and will always be grateful to my advisors Chris Buckley and Anil Seth
for all their support and advice; for giving me space to be confused, trust to
look for my own path, and freedom to pursue it. Your combined wisdom has been
invaluable in shaping my research and my growth as a scientist.

I would also like to express my deepest gratitude to my other mentors. My
internship with Hady Elsahar, Germán Kruszewski, and Marc Dymetman was a
pivotal moment in my development as a machine learning researcher. They taught
me the importance of mathematical rigor and had a tremendous lasting impact on
way I think about language models. Half of the material for this thesis was
born during this critical period, and I am forever indebted to their guidance.

Finally, I am eternally grateful to Ethan Perez, the single most influential
person in my research journey. Ethan spent countless hours discussing
experiments with me, teaching me how to write about them convincingly. He
always pushed me to ask the right questions and doing research that matters
for the most important challenges humanity will soon face. He guided me
towards the most impactful work I have ever done, and I will always cherish
his mentorship and friendship. Any future impact of my work is a testament to
the impact of Ethan’s guidance.

My work wouldn’t have been possible without my collaborators: Mikita Balesni,
Lukas Berglund, Rasika Bhalerao, Sam Bowman, Stephen Casper, Jun Shern Chan,
Lawrence Chan, Angelica Chen, Kyunghyun Cho, Xander Davies, David Duvenaud,
Owain Evans, Dongyoung Go, Jérémy Scheurer, Max Kaufmann, Daniel Kokotajlo,
Łukasz Kuciński, Jason Phang, Jos Rozen, Richard Ren, Mrinank Sharma, Kejian
Shi, Asa Cooper Stickland, Meg Tong, and Julian Zubek. Their contributions and
ideas have been invaluable.

In addition to those I directly collaborated with, I’m also grateful to all
the people who influenced me through many stimulating conversations and whose
vision and work inspired me and shaped my own research in innumerable ways.
These include David Bau, Herbie Bradley, Daniel Braun, Alex Cloud, Ajeya
Cotra, Paul Christiano, David Dohan, Scott Emmons, Adam Gleave, Leo Gao, Karol
Hausman, Marius Hobbhahn, Francesco Innocenti, Geoffrey Irving, Michael
Janner, Sebastian Jaszczur, Holden Karnofsky, David Krueger, David Lindner,
Kyle McDonell, Ian McKenzie, Beren Millidge, Joe Murray, Neel Nanda, Jacob
Pfau, Nadine Spychala, Matt Reardon, David Rein, Laria Reynolds, Nina Rimsky,
Wiktor Rorot, Claudia Shi, Dane Sherburn, Buck Shlegeris, Ivor Simpson,
Charlie Snell, Alec Tschantz, Miles Turpin, Sean Welleck, and Daniel Ziegler.
I’m also grateful to the AI safety community at large and operations teams of
Constellation in Berkeley and London Initiative for Safe AI who hosted me at
various stages of my PhD.

I would not be the researcher I am without the mentorship I received before my
PhD. Joanna Rączaszek-Leonardi showed me the value of open, slow and kind
science; Piotr Miłoś patiently guided me throughout my first serious
experiments and Marcin Miłkowski taught me how to express my thoughts clearly.
Their early guidance laid the foundation for everything that followed and will
follow.

I’m also grateful to the Leverhulme Trust, OpenPhilantropy and the Long-Term
Future Fund for providing financial support to my research, enabling me to
pursue ideas without constraints.

Finally, I’m grateful for all the personal support I’ve received from my
parents, friends, and close ones. Their unwavering encouragement and loving
understanding have been a constant source of strength throughout my PhD
journey.

I, Tomasz Korbak, hereby declare that this thesis has not been and will not
be, submitted in whole or in part to another university for the award of any
other degree.

Brighton,

18 Sept 2023

Tomasz Korbak

University of Sussex

Faculty of Engineering and Informatics

Department of Informatics

Doctoral Thesis

Aligning language models
with human preferences

by Tomasz Korbak

Abstract

Language models (LMs) trained on vast quantities of text data can acquire
sophisticated skills such as generating summaries, answering questions or
generating code. However, they also manifest behaviors that violate human
preferences, e.g., they can generate offensive content, falsehoods or
perpetuate social biases. In this thesis, I explore several approaches to
aligning LMs with human preferences. First, I argue that aligning LMs can be
seen as Bayesian inference: conditioning a prior (base, pretrained LM) on
evidence about human preferences (Chapter 2). Conditioning on human
preferences can be implemented in numerous ways. In Chapter 3, I investigate
the relation between two approaches to finetuning pretrained LMs using
feedback given by a scoring function: reinforcement learning from human
feedback (RLHF) and distribution matching. I show that RLHF can be seen as a
special case of distribution matching but distributional matching is strictly
more general. In chapter 4, I show how to extend the distribution matching to
conditional language models. Finally, in chapter 5 I explore a different root:
conditioning an LM on human preferences already during pretraining. I show
that involving human feedback from the very start tends to be more effective
than using it only during supervised finetuning. Overall, these results
highlight the room for alignment techniques different from and complementary
to RLHF.

Keywords: language models, reinforcement learning, alignment, finetuning,
pretraining, AI safety, human feedback

hansen

*