* Hutter et al. [2012] M. Hutter, C. Gehring, M. Bloesch, M. A. Hoepflinger, C. D. Remy, and R. Siegwart.  Starleth: A compliant quadrupedal robot for fast, efficient, and versatile locomotion.  In _Adaptive mobile robotics_ , pages 483–490. World Scientific, 2012\.
  * Hutter et al. [2014] M. Hutter, H. Sommer, C. Gehring, M. Hoepflinger, M. Bloesch, and R. Siegwart.  Quadrupedal locomotion using hierarchical operational space control.  _The International Journal of Robotics Research_ , 33(8):1047–1062, 2014.
  * Hutter et al. [2016] M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V. Tsounis, J. Hwangbo, K. Bodie, P. Fankhauser, M. Bloesch, et al.  Anymal-a highly mobile and dynamic quadrupedal robot.  In _2016 IEEE/RSJ international conference on intelligent robots and systems (IROS)_ , pages 38–44. IEEE, 2016.
  * Hwangbo et al. [2018] J. Hwangbo, J. Lee, and M. Hutter.  Per-contact iteration method for solving contact dynamics.  _IEEE Robotics and Automation Letters_ , 3(2):895–902, 2018.
  * Hwangbo et al. [2019] J. Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter.  Learning agile and dynamic motor skills for legged robots.  _Science Robotics_ , 4(26):eaau5872, 2019.
  * Ibarz et al. [2021] J. Ibarz, J. Tan, C. Finn, M. Kalakrishnan, P. Pastor, and S. Levine.  How to train your robot with deep reinforcement learning: lessons we have learned.  _The International Journal of Robotics Research_ , 40(4-5):698–721, 2021.
  * Ijspeert [2001] A. J. Ijspeert.  A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander.  _Biological cybernetics_ , 84(5):331–348, 2001\.
  * Ijspeert [2008] A. J. Ijspeert.  Central pattern generators for locomotion control in animals and robots: a review.  _Neural networks_ , 21(4):642–653, 2008.
  * Iscen et al. [2018] A. Iscen, K. Caluwaerts, J. Tan, T. Zhang, E. Coumans, V. Sindhwani, and V. Vanhoucke.  Policies modulating trajectory generators.  In _Conference on Robot Learning_ , pages 916–926. PMLR, 2018.
  * Janner et al. [2019] M. Janner, J. Fu, M. Zhang, and S. Levine.  When to trust your model: Model-based policy optimization.  _Advances in neural information processing systems_ , 32, 2019.
  * Jenelten et al. [2024] F. Jenelten, J. He, F. Farshidian, and M. Hutter.  Dtc: Deep tracking control.  _Science Robotics_ , 9(86):eadh5401, 2024.
  * Jeon et al. [2023] S. Jeon, M. Jung, S. Choi, B. Kim, and J. Hwangbo.  Learning whole-body manipulation for quadrupedal robot.  _arXiv preprint arXiv:2308.16820_ , 2023.
  * Ji et al. [2022a] G. Ji, J. Mun, H. Kim, and J. Hwangbo.  Concurrent training of a control policy and a state estimator for dynamic and robust legged locomotion.  _IEEE Robotics and Automation Letters_ , 7(2):4630–4637, 2022a.
  * Ji et al. [2022b] Y. Ji, Z. Li, Y. Sun, X. B. Peng, S. Levine, G. Berseth, and K. Sreenath.  Hierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 1479–1486. IEEE, 2022b.
  * Ji et al. [2023] Y. Ji, G. B. Margolis, and P. Agrawal.  Dribblebot: Dynamic legged manipulation in the wild.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 5155–5162. IEEE, 2023.
  * Johannink et al. [2019] T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine.  Residual reinforcement learning for robot control.  In _2019 International Conference on Robotics and Automation (ICRA)_ , pages 6023–6029. IEEE, 2019.
  * Johnson et al. [2017] M. Johnson, B. Shrewsbury, S. Bertrand, D. Calvert, T. Wu, D. Duran, D. Stephen, N. Mertins, J. Carff, W. Rifenburgh, et al.  Team ihmc’s lessons learned from the darpa robotics challenge: Finding data in the rubble.  _Journal of Field Robotics_ , 34(2):241–261, 2017\.
  * Junyao et al. [2013] G. Junyao, D. Xingguang, H. Qiang, L. Huaxin, X. Zhe, L. Yi, L. Xin, and S. Wentao.  The research of hydraulic quadruped bionic robot design.  In _2013 ICME international conference on complex medical engineering_ , pages 620–625. IEEE, 2013.
  * Kajita et al. [2001] S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa.  The 3d linear inverted pendulum mode: A simple modeling for a biped walking pattern generation.  In _Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180)_ , volume 1, pages 239–246. IEEE, 2001.
  * Kajita et al. [2003] S. Kajita, F. Kanehiro, K. Kaneko, K. Fujiwara, K. Harada, K. Yokoi, and H. Hirukawa.  Biped walking pattern generation by using preview control of zero-moment point.  In _2003 IEEE international conference on robotics and automation (Cat. No. 03CH37422)_ , volume 2, pages 1620–1626. IEEE, 2003.
  * Kalakrishnan et al. [2011] M. Kalakrishnan, J. Buchli, P. Pastor, M. Mistry, and S. Schaal.  Learning, planning, and control for quadruped locomotion over challenging terrain.  _The International Journal of Robotics Research_ , 30(2):236–258, 2011.
  * Kang et al. [2021] D. Kang, S. Zimmermann, and S. Coros.  Animal gaits on quadrupedal robots using motion matching and model-based control. in 2021 ieee.  In _RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE_ , 2021.
  * Kang et al. [2023] D. Kang, J. Cheng, M. Zamora, F. Zargarbashi, and S. Coros.  Rl+ model-based control: Using on-demand optimal control to learn versatile legged locomotion.  _IEEE Robotics and Automation Letters_ , 2023.
  * Katz et al. [2019] B. Katz, J. Di Carlo, and S. Kim.  Mini cheetah: A platform for pushing the limits of dynamic quadruped control.  In _2019 international conference on robotics and automation (ICRA)_ , pages 6295–6301. IEEE, 2019.
  * Kau et al. [2019] N. Kau, A. Schultz, N. Ferrante, and P. Slade.  Stanford doggo: An open-source, quasi-direct-drive quadruped.  In _2019 International conference on robotics and automation (ICRA)_ , pages 6309–6315. IEEE, 2019.
  * Khadiv et al. [2019] M. Khadiv, S. A. A. Moosavian, A. Yousefi-Koma, M. Sadedel, A. Ehsani-Seresht, and S. Mansouri.  Rigid vs compliant contact: an experimental study on biped walking.  _Multibody System Dynamics_ , 45:379–401, 2019.
  * Khadiv et al. [2023] M. Khadiv, A. Meduri, H. Zhu, L. Righetti, and B. Schölkopf.  Learning locomotion skills from mpc in sensor space.  In _Learning for Dynamics and Control Conference_. PMLR, 2023.
  * Kim et al. [2023a] J. Kim, J. Lee, and A. D. Ames.  Safety-critical coordination for cooperative legged locomotion via control barrier functions.  In _2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 2368–2375. IEEE, 2023a.
  * Kim et al. [2023b] J. Kim, T. Li, and S. Ha.  Armp: Autoregressive motion planning for quadruped locomotion and navigation in complex indoor environments.  In _2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 2731–2737. IEEE, 2023b.
  * Kim et al. [2022] S. Kim, M. Sorokin, J. Lee, and S. Ha.  Humanconquad: human motion control of quadrupedal robots using deep reinforcement learning.  In _SIGGRAPH Asia 2022 Emerging Technologies_ , pages 1–2, 2022.
  * Kim et al. [2021] T. Kim, M. Jang, and J. Kim.  A survey on simulation environments for reinforcement learning.  In _2021 18th International Conference on Ubiquitous Robots (UR)_ , pages 63–67. IEEE, 2021.
  * Kim et al. [2023c] Y. Kim, H. Oh, J. Lee, J. Choi, G. Ji, M. Jung, D. Youm, and J. Hwangbo.  Not only rewards but also constraints: Applications on legged robot locomotion.  _arXiv preprint arXiv:2308.12517_ , 2023c.
  * Kira [2022] Z. Kira.  Awesome-llm-robotics, 2022.  URL https://github.com/GT-RIPL/Awesome-LLM-Robotics.
  * Klipfel et al. [2023] A. Klipfel, N. Sontakke, R. Liu, and S. Ha.  Learning a single policy for diverse behaviors on a quadrupedal robot using scalable motion imitation.  In _2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 2768–2775. IEEE, 2023.
  * Koenemann et al. [2015] J. Koenemann, A. Del Prete, Y. Tassa, E. Todorov, O. Stasse, M. Bennewitz, and N. Mansard.  Whole-body model-predictive control applied to the hrp-2 humanoid.  In _2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 3346–3351. IEEE, 2015.
  * Kohl and Stone [2004] N. Kohl and P. Stone.  Policy gradient reinforcement learning for fast quadrupedal locomotion.  In _IEEE International Conference on Robotics and Automation, 2004\. Proceedings. ICRA’04. 2004_ , volume 3, pages 2619–2624. IEEE, 2004.
  * Kovalev et al. [2023] V. Kovalev, A. Shkromada, H. Ouerdane, and P. Osinenko.  Combining model-predictive control and predictive reinforcement learning for stable quadrupedal robot locomotion.  _arXiv preprint arXiv:2307.07752_ , 2023.
  * Kuindersma et al. [2016] S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion, and R. Tedrake.  Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot.  _Autonomous robots_ , 40:429–455, 2016.
  * Kumar et al. [2021] A. Kumar, Z. Fu, D. Pathak, and J. Malik.  Rma: Rapid motor adaptation for legged robots.  _arXiv preprint arXiv:2107.04034_ , 2021.
  * Kumar et al. [2023] K. N. Kumar, I. Essa, and S. Ha.  Cascaded compositional residual learning for complex interactive behaviors.  _IEEE Robotics and Automation Letters_ , 2023.
  * Kwon et al. [2020] T. Kwon, Y. Lee, and M. Van De Panne.  Fast and flexible multilegged locomotion using learned centroidal dynamics.  _ACM Transactions on Graphics (TOG)_ , 39(4):46–1, 2020.
  * Le Lidec et al. [2021] Q. Le Lidec, I. Kalevatykh, I. Laptev, C. Schmid, and J. Carpentier.  Differentiable simulation for physical system identification.  _IEEE Robotics and Automation Letters_ , 6(2):3413–3420, 2021.
  * LeCun et al. [2015] Y. LeCun, Y. Bengio, and G. Hinton.  Deep learning.  _nature_ , 521(7553):436–444, 2015.
  * Lee et al. [2018a] G. Lee, B. Hou, A. Mandalika, J. Lee, S. Choudhury, and S. S. Srinivasa.  Bayesian policy optimization for model uncertainty.  _arXiv preprint arXiv:1810.01014_ , 2018a.
  * Lee et al. [2018b] J. Lee, M. X. Grey, S. Ha, T. Kunz, S. Jain, Y. Ye, S. S. Srinivasa, M. Stilman, and C. Karen Liu.  Dart: Dynamic animation and robotics toolkit.  _The Journal of Open Source Software_ , 3(22):500, 2018b.
  * Lee et al. [2020] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter.  Learning quadrupedal locomotion over challenging terrain.  _Science robotics_ , 5(47):eabc5986, 2020.
  * Lee et al. [2023] J. Lee, L. Schroth, V. Klemm, M. Bjelonic, A. Reske, and M. Hutter.  Evaluation of constrained reinforcement learning algorithms for legged locomotion.  _arXiv preprint arXiv:2309.15430_ , 2023.
  * Lee et al. [2024] J. Lee, M. Bjelonic, A. Reske, L. Wellhausen, T. Miki, and M. Hutter.  Learning robust autonomous navigation and locomotion for wheeled-legged robots.  _Science Robotics_ , 9(89):eadi9641, 2024.  10.1126/scirobotics.adi9641.
  * Lee et al. [2022] K.-H. Lee, O. Nachum, T. Zhang, S. Guadarrama, J. Tan, and W. Yu.  Pi-ars: Accelerating evolution-learned visual-locomotion with predictive information representations.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 1447–1454. IEEE, 2022.
  * Lembono et al. [2020] T. S. Lembono, C. Mastalli, P. Fernbach, N. Mansard, and S. Calinon.  Learning how to walk: Warm-starting optimal control solver with memory of motion.  In _2020 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1357–1363. IEEE, 2020.
  * Lengagne et al. [2013] S. Lengagne, J. Vaillant, E. Yoshida, and A. Kheddar.  Generation of whole-body optimal dynamic multi-contact motions.  _The International Journal of Robotics Research_ , 32(9-10):1104–1119, 2013.
  * letter [2022] letter.  General purpose robots should not be weaponized: An open letter to the robotics industry and our communities, 2022.  URL https://shorturl.at/f9LhG.
  * Li et al. [2023a] C. Li, S. Blaes, P. Kolev, M. Vlastelica, J. Frey, and G. Martius.  Versatile skill control via self-supervised adversarial imitation of unlabeled mixed motions.  In _2023 IEEE international conference on robotics and automation (ICRA)_ , pages 2944–2950. IEEE, 2023a.
  * Li et al. [2023b] C. Li, M. Vlastelica, S. Blaes, J. Frey, F. Grimminger, and G. Martius.  Learning agile skills via adversarial imitation of rough partial demonstrations.  In _Conference on Robot Learning_ , pages 342–352. PMLR, 2023b.
  * Li et al. [2021a] H. Li, R. J. Frei, and P. M. Wensing.  Model hierarchy predictive control of robotic systems.  _IEEE Robotics and Automation Letters_ , 6(2):3373–3380, 2021a.
  * Li et al. [2021b] T. Li, R. Calandra, D. Pathak, Y. Tian, F. Meier, and A. Rai.  Planning in learned latent action spaces for generalizable legged locomotion.  _IEEE Robotics and Automation Letters_ , 6(2):2682–2689, 2021b.
  * Li et al. [2023c] T. Li, H. Jung, M. Gombolay, Y. K. Cho, and S. Ha.  Crossloco: Human motion driven control of legged robots via guided unsupervised reinforcement learning.  _arXiv preprint arXiv:2309.17046_ , 2023c.
  * Li et al. [2023d] T. Li, J. Won, J. Cho, S. Ha, and A. Rai.  Fastmimic: Model-based motion imitation for agile, diverse and generalizable quadrupedal locomotion.  _Robotics_ , 12(3):90, 2023d.
  * Li et al. [2024] Z. Li, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath.  Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control.  _arXiv preprint arXiv:2401.16889_ , 2024.
  * Liang et al. [2024] J. Liang, F. Xia, W. Yu, A. Zeng, M. G. Arenas, M. Attarian, M. Bauza, M. Bennice, A. Bewley, A. Dostmohamed, et al.  Learning to learn faster from human feedback with language model predictive control.  _arXiv preprint arXiv:2402.11450_ , 2024.
  * Lidec et al. [2022] Q. L. Lidec, L. Montaut, C. Schmid, I. Laptev, and J. Carpentier.  Augmenting differentiable physics with randomized smoothing.  _arXiv preprint arXiv:2206.11884_ , 2022.
  * Lidec et al. [2023] Q. L. Lidec, W. Jallet, L. Montaut, I. Laptev, C. Schmid, and J. Carpentier.  Contact models in robotics: a comparative analysis.  _arXiv preprint arXiv:2304.06372_ , 2023.
  * Lillicrap et al. [2015] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra.  Continuous control with deep reinforcement learning.  _arXiv preprint arXiv:1509.02971_ , 2015.
  * Liu and Negrut [2021] C. K. Liu and D. Negrut.  The role of physics-based simulators in robotics.  _Annual Review of Control, Robotics, and Autonomous Systems_ , 4:35–58, 2021.
  * Liu et al. [2024] M. Liu, Z. Chen, X. Cheng, Y. Ji, R. Yang, and X. Wang.  Visual whole-body control for legged loco-manipulation.  _arXiv preprint arXiv:2403.16967_ , 2024.
  * Liu et al. [2022a] Y. Liu, J. Shen, J. Zhang, X. Zhang, T. Zhu, and D. Hong.  Design and control of a miniature bipedal robot with proprioceptive actuation for dynamic behaviors.  In _2022 International Conference on Robotics and Automation (ICRA)_ , pages 8547–8553. IEEE, 2022a.
  * Liu et al. [2022b] Z. Liu, Z. Cen, V. Isenbaev, W. Liu, S. Wu, B. Li, and D. Zhao.  Constrained variational policy optimization for safe reinforcement learning.  In _International Conference on Machine Learning_ , pages 13644–13668. PMLR, 2022b.
  * Luck et al. [2017] K. S. Luck, J. Campbell, M. A. Jansen, D. M. Aukes, and H. B. Amor.  From the lab to the desert: Fast prototyping and learning of robot locomotion.  _arXiv preprint arXiv:1706.01977_ , 2017.
  * Lykov et al. [2024] A. Lykov, M. Litvinov, M. Konenkov, R. Prochii, N. Burtsev, A. A. Abdulkarim, A. Bazhenov, V. Berman, and D. Tsetserukou.  Cognitivedog: Large multimodal model based system to translate vision and language into action of quadruped robot.  _arXiv preprint arXiv:2401.09388_ , 2024.
  * Ma et al. [2024] Y. J. Ma, W. Liang, H. Wang, S. Wang, Y. Zhu, L. Fan, O. Bastani, and D. Jayaraman.  Dreureka: Language model guided sim-to-real transfer.  In _Proceedings of Robotics: Science and Systems_ , 2024.
  * Makoviychuk et al. [2021] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al.  Isaac gym: High performance gpu-based physics simulation for robot learning.  _arXiv preprint arXiv:2108.10470_ , 2021.
  * Mania et al. [2018] H. Mania, A. Guy, and B. Recht.  Simple random search of static linear policies is competitive for reinforcement learning.  _Advances in neural information processing systems_ , 31, 2018.
  * Marco et al. [2021] A. Marco, D. Baumann, M. Khadiv, P. Hennig, L. Righetti, and S. Trimpe.  Robot learning with crash constraints.  _IEEE Robotics and Automation Letters_ , 6(2):1439–1446, 2021.
  * Margolis and Agrawal [2023] G. B. Margolis and P. Agrawal.  Walk these ways: Tuning robot control for generalization with multiplicity of behavior.  In _Conference on Robot Learning_ , pages 22–31. PMLR, 2023.
  * Margolis et al. [2021] G. B. Margolis, T. Chen, K. Paigwar, X. Fu, D. Kim, S. Kim, and P. Agrawal.  Learning to jump from pixels.  _arXiv preprint arXiv:2110.15344_ , 2021.
  * Margolis et al. [2023] G. B. Margolis, X. Fu, Y. Ji, and P. Agrawal.  Learning to see physical properties with active sensing motor policies.  _arXiv preprint arXiv:2311.01405_ , 2023.
  * Margolis et al. [2024] G. B. Margolis, G. Yang, K. Paigwar, T. Chen, and P. Agrawal.  Rapid locomotion via reinforcement learning.  _The International Journal of Robotics Research_ , 43(4):572–587, 2024.
  * Marhefka and Orin [1996] D. W. Marhefka and D. E. Orin.  Simulation of contact using a nonlinear damping model.  In _Proceedings of IEEE international conference on robotics and automation_ , volume 2, pages 1662–1668. IEEE, 1996.
  * Mastalli et al. [2023] C. Mastalli, W. Merkt, G. Xin, J. Shim, M. Mistry, I. Havoutis, and S. Vijayakumar.  Agile maneuvers in legged robots: a predictive control approach.  _IEEE Transactions on Robotics_ , 2023.
  * Masterjohn et al. [2022] J. Masterjohn, D. Guoy, J. Shepherd, and A. Castro.  Velocity level approximation of pressure field contact patches.  _IEEE Robotics and Automation Letters_ , 7(4):11593–11600, 2022.
  * Meduri et al. [2023] A. Meduri, P. Shah, J. Viereck, M. Khadiv, I. Havoutis, and L. Righetti.  Biconmp: A nonlinear model predictive control framework for whole body motion planning.  _IEEE Transactions on Robotics_ , 2023.
  * Miki et al. [2022a] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter.  Learning robust perceptive locomotion for quadrupedal robots in the wild.  _Science Robotics_ , 7(62):eabk2822, 2022a.
  * Miki et al. [2022b] T. Miki, L. Wellhausen, R. Grandia, F. Jenelten, T. Homberger, and M. Hutter.  Elevation mapping for locomotion and navigation using gpu.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 2273–2280. IEEE, 2022b.
  * Miki et al. [2024] T. Miki, J. Lee, L. Wellhausen, and M. Hutter.  Learning to walk in confined spaces using 3d representation.  _arXiv preprint arXiv:2403.00187_ , 2024.
  * Mistry et al. [2010] M. Mistry, J. Buchli, and S. Schaal.  Inverse dynamics control of floating base systems using orthogonal decomposition.  In _2010 IEEE international conference on robotics and automation_ , pages 3406–3412. IEEE, 2010.
  * MJX [2023] MJX.  Mujoco3, 2023.  URL https://mujoco.readthedocs.io/en/stable/mjx.html.
  * Mnih et al. [2015] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al.  Human-level control through deep reinforcement learning.  _nature_ , 518(7540):529–533, 2015.
  * Mnih et al. [2016] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu.  Asynchronous methods for deep reinforcement learning.  In _International conference on machine learning_ , pages 1928–1937. PMLR, 2016.
  * Mordatch et al. [2012] I. Mordatch, E. Todorov, and Z. Popović.  Discovery of complex behaviors through contact-invariant optimization.  _ACM Transactions on Graphics (ToG)_ , 31(4):1–8, 2012.
  * Morimoto and Atkeson [2009] J. Morimoto and C. G. Atkeson.  Nonparametric representation of an approximated poincaré map for learning biped locomotion.  _Autonomous Robots_ , 27(2):131–144, 2009.
  * Morimoto et al. [2005] J. Morimoto, J. Nakanishi, G. Endo, G. Cheng, C. G. Atkeson, and G. Zeglin.  Poincare-map-based reinforcement learning for biped walking.  In _Proceedings of the 2005 IEEE International Conference on Robotics and Automation_ , pages 2381–2386. IEEE, 2005.
  * Müller [2020] V. C. Müller.  Ethics of artificial intelligence and robotics.  _Stanford Encyclopedia of Philosophy_ , 2020.
  * Muratore et al. [2021] F. Muratore, C. Eilers, M. Gienger, and J. Peters.  Data-efficient domain randomization with bayesian optimization.  _IEEE Robotics and Automation Letters_ , 6(2):911–918, 2021.
  * Murphy [2015] R. R. Murphy.  Meta-analysis of autonomy at the darpa robotics challenge trials.  _Journal of Field Robotics_ , 32(2):189–191, 2015\.
  * Nagabandi et al. [2018] A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine.  Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning.  In _2018 IEEE international conference on robotics and automation (ICRA)_ , pages 7559–7566. IEEE, 2018.
  * Nahrendra et al. [2023] I. M. A. Nahrendra, B. Yu, and H. Myung.  Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 5078–5084. IEEE, 2023.
  * Nelson et al. [2018] G. Nelson, A. Saunders, and R. Playter.  _The PETMAN and Atlas Robots at Boston Dynamics_ , pages 1–18.  Springer Netherlands, Dordrecht, 2018.  ISBN 978-94-007-7194-9.  10.1007/978-94-007-7194-9_15-1.  URL https://doi.org/10.1007/978-94-007-7194-9_15-1.
  * Neunert et al. [2016] M. Neunert, C. De Crousaz, F. Furrer, M. Kamel, F. Farshidian, R. Siegwart, and J. Buchli.  Fast nonlinear model predictive control for unified trajectory optimization and tracking.  In _2016 IEEE international conference on robotics and automation (ICRA)_ , pages 1398–1404. IEEE, 2016.
  * Neunert et al. [2018] M. Neunert, M. Stäuble, M. Giftthaler, C. D. Bellicoso, J. Carius, C. Gehring, M. Hutter, and J. Buchli.  Whole-body nonlinear model predictive control through contacts for quadrupeds.  _IEEE Robotics and Automation Letters_ , 3(3):1458–1465, 2018.
  * Oleynikova et al. [2017] H. Oleynikova, Z. Taylor, M. Fehr, R. Siegwart, and J. Nieto.  Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning.  In _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 1366–1373. IEEE, 2017.
  * Omar et al. [2023] S. Omar, L. Amatucci, V. Barasuol, G. Turrisi, and C. Semini.  Safesteps: Learning safer footstep planning policies for legged robots via model-based priors.  In _2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids)_ , pages 1–8. IEEE, 2023.
  * Pandala et al. [2022] A. Pandala, R. T. Fawcett, U. Rosolia, A. D. Ames, and K. A. Hamed.  Robust predictive control for quadrupedal locomotion: Learning to close the gap between reduced-and full-order models.  _IEEE Robotics and Automation Letters_ , 7(3):6622–6629, 2022.
  * Pang et al. [2023] T. Pang, H. T. Suh, L. Yang, and R. Tedrake.  Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models.  _IEEE Transactions on Robotics_ , 2023.
  * Papadopoulos and Buehler [2000] D. Papadopoulos and M. Buehler.  Stable running in a quadruped robot with compliant legs.  In _Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065)_ , volume 1, pages 444–449. IEEE, 2000.
  * Park et al. [2015] H.-W. Park, P. M. Wensing, S. Kim, et al.  Online planning for autonomous running jumps over obstacles in high-speed quadrupeds.  In _Proceedings of Robotics: Science and Systems_ , 2015.
  * Peng and Van De Panne [2017] X. B. Peng and M. Van De Panne.  Learning locomotion skills using deeprl: Does the choice of action space matter?  In _Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation_ , pages 1–13, 2017.
  * Peng et al. [2017] X. B. Peng, G. Berseth, K. Yin, and M. Van De Panne.  Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning.  _Acm transactions on graphics (tog)_ , 36(4):1–13, 2017.
  * Peng et al. [2018] X. B. Peng, P. Abbeel, S. Levine, and M. Van de Panne.  Deepmimic: Example-guided deep reinforcement learning of physics-based character skills.  _ACM Transactions On Graphics (TOG)_ , 37(4):1–14, 2018.
  * Peng et al. [2020] X. B. Peng, E. Coumans, T. Zhang, T.-W. Lee, J. Tan, and S. Levine.  Learning agile robotic locomotion skills by imitating animals.  _arXiv preprint arXiv:2004.00784_ , 2020.
  * Peng et al. [2022] X. B. Peng, Y. Guo, L. Halper, S. Levine, and S. Fidler.  Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters.  _ACM Transactions On Graphics (TOG)_ , 41(4):1–17, 2022.
  * Pham et al. [2018] Q. Pham, R. Madhavan, L. Righetti, W. Smart, and R. Chatila.  The impact of robotics and automation on working conditions and employment.  _IEEE Robotics & Automation Magazine_, 25(2):126–128, 2018.
  * Pomerleau [1988] D. A. Pomerleau.  Alvinn: An autonomous land vehicle in a neural network.  _Advances in neural information processing systems_ , 1, 1988.
  * Ponton et al. [2014] B. Ponton, F. Farshidian, and J. Buchli.  Learning compliant locomotion on a quadruped robot.  In _IROS Workshop (Ed.), Compliant manipulation: Challenges in learning and control_. Citeseer, 2014.
  * Ponton et al. [2021] B. Ponton, M. Khadiv, A. Meduri, and L. Righetti.  Efficient multicontact pattern generation with sequential convex approximations of the centroidal dynamics.  _IEEE Transactions on Robotics_ , 37(5):1661–1679, 2021.
  * Posa et al. [2014] M. Posa, C. Cantu, and R. Tedrake.  A direct method for trajectory optimization of rigid bodies through contact.  _The International Journal of Robotics Research_ , 33(1):69–81, 2014.
  * Pratt et al. [1997] J. Pratt, P. Dilworth, and G. Pratt.  Virtual model control of a bipedal walking robot.  In _Proceedings of international conference on robotics and automation_ , volume 1, pages 193–198. IEEE, 1997.
  * Radosavovic et al. [2024] I. Radosavovic, T. Xiao, B. Zhang, T. Darrell, J. Malik, and K. Sreenath.  Real-world humanoid locomotion with reinforcement learning.  _Science Robotics_ , 9(89):eadi9579, 2024.
  * Raibert [1984] M. H. Raibert.  Hopping in legged systems—modeling and simulation for the two-dimensional one-legged case.  _IEEE Transactions on Systems, Man, and Cybernetics_ , pages 451–463, 1984.
  * Raibert [1986] M. H. Raibert.  _Legged robots that balance_.  MIT press, 1986.
  * Raibert et al. [2008] M. H. Raibert, K. Blankespoor, G. M. Nelson, and R. Playter.  Bigdog, the rough-terrain quadruped robot.  _IFAC Proceedings Volumes_ , 41:10822–10825, 2008.  URL https://api.semanticscholar.org/CorpusID:8414062.
  * Raković and Levine [2018] S. V. Raković and W. S. Levine.  _Handbook of model predictive control_.  Springer, 2018.
  * Ravi et al. [2024] A. K. C. Ravi, V. Dhédin, A. Jordana, H. Zhu, A. Meduri, L. Righetti, B. Schölkopf, and M. Khadiv.  Efficient search and learning for agile locomotion on stepping stones.  _arXiv preprint arXiv:2403.03639_ , 2024.
  * Reske et al. [2021] A. Reske, J. Carius, Y. Ma, F. Farshidian, and M. Hutter.  Imitation learning from mpc for quadrupedal multi-gait control.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 5014–5020. IEEE, 2021.
  * Righetti et al. [2013] L. Righetti, J. Buchli, M. Mistry, M. Kalakrishnan, and S. Schaal.  Optimal distribution of contact forces with inverse-dynamics control.  _The International Journal of Robotics Research_ , 32(3):280–298, 2013.
  * Righetti et al. [2018] L. Righetti, Q.-C. Pham, R. Madhavan, and R. Chatila.  Lethal autonomous weapon systems [ethical, legal, and societal issues].  _IEEE Robotics & Automation Magazine_, 25(1):123–126, 2018.
  * Rong et al. [2012] X. Rong, Y. Li, J. Ruan, and B. Li.  Design and simulation for a hydraulic actuated quadruped robot.  _Journal of mechanical science and technology_ , 26:1171–1177, 2012.
  * Ross et al. [2011] S. Ross, G. Gordon, and D. Bagnell.  A reduction of imitation learning and structured prediction to no-regret online learning.  In _Proceedings of the fourteenth international conference on artificial intelligence and statistics_ , pages 627–635. JMLR Workshop and Conference Proceedings, 2011.
  * Rudin et al. [2022a] N. Rudin, D. Hoeller, M. Bjelonic, and M. Hutter.  Advanced skills by learning locomotion and local navigation end-to-end.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 2497–2503. IEEE, 2022a.
  * Rudin et al. [2022b] N. Rudin, D. Hoeller, P. Reist, and M. Hutter.  Learning to walk in minutes using massively parallel deep reinforcement learning.  In _Conference on Robot Learning_ , pages 91–100. PMLR, 2022b.
  * Rummery and Niranjan [1994] G. A. Rummery and M. Niranjan.  _On-line Q-learning using connectionist systems_ , volume 37.  University of Cambridge, Department of Engineering Cambridge, UK, 1994\.
  * Ruppert and Badri-Spröwitz [2022] F. Ruppert and A. Badri-Spröwitz.  Learning plastic matching of robot dynamics in closed-loop central pattern generators.  _Nature Machine Intelligence_ , 4(7):652–660, 2022.
  * Saggar et al. [2007] M. Saggar, T. D’Silva, N. Kohl, and P. Stone.  Autonomous learning of stable quadruped locomotion.  In _RoboCup 2006: Robot Soccer World Cup X 10_ , pages 98–109. Springer, 2007.
  * Sarmadi et al. [2023] A. Sarmadi, P. Krishnamurthy, and F. Khorrami.  High-dimensional controller tuning through latent representations.  _arXiv preprint arXiv:2309.12487_ , 2023.
  * Schaal [2009] S. Schaal.  The sl simulation and real-time control software package.  Technical report, Citeseer, 2009.
  * Schaal [2018] S. Schaal.  _Historical Perspective of Humanoid Robot Research in the Americas_ , pages 1–9.  Springer Netherlands, Dordrecht, 2018.  ISBN 978-94-007-7194-9.  10.1007/978-94-007-7194-9_143-1.  URL https://doi.org/10.1007/978-94-007-7194-9_143-1.
  * Schuitema et al. [2010] E. Schuitema, M. Wisse, T. Ramakers, and P. Jonker.  The design of leo: a 2d bipedal walking robot for online autonomous reinforcement learning.  In _2010 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 3238–3243. IEEE, 2010.
  * Schulman et al. [2015] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz.  Trust region policy optimization.  In _International conference on machine learning_ , pages 1889–1897. PMLR, 2015.
  * Schulman et al. [2017] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ , 2017.
  * Schwarke et al. [2023] C. Schwarke, V. Klemm, M. Van der Boon, M. Bjelonic, and M. Hutter.  Curiosity-driven learning of joint locomotion and manipulation tasks.  In _Proceedings of The 7th Conference on Robot Learning_ , volume 229, pages 2594–2610. PMLR, 2023.
  * Schwarke et al. [2024] C. Schwarke, V. Klemm, J. Tordesillas, J.-P. Sleiman, and M. Hutter.  Learning quadrupedal locomotion via differentiable simulation.  _arXiv preprint arXiv:2404.02887_ , 2024.
  * Semini [2010] C. Semini.  Hyq-design and development of a hydraulically actuated quadruped robot.  _Doctor of Philosophy (Ph. D.), University of Genoa, Italy_ , 2010\.
  * Semini et al. [2016] C. Semini, V. Barasuol, J. Goldsmith, M. Frigerio, M. Focchi, Y. Gao, and D. G. Caldwell.  Design of the hydraulically actuated, torque-controlled quadruped robot hyq2max.  _IEEE/Asme Transactions on Mechatronics_ , 22(2):635–646, 2016.
  * Seok et al. [2014] S. Seok, A. Wang, M. Y. Chuah, D. J. Hyun, J. Lee, D. M. Otten, J. H. Lang, and S. Kim.  Design principles for energy-efficient legged locomotion and implementation on the mit cheetah robot.  _Ieee/asme transactions on mechatronics_ , 20(3):1117–1129, 2014.
  * Shafiee et al. [2023] M. Shafiee, G. Bellegarda, and A. Ijspeert.  Puppeteer and marionette: Learning anticipatory quadrupedal locomotion based on interactions of a central pattern generator and supraspinal drive.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1112–1119. IEEE, 2023.
  * Sharma et al. [2020] A. Sharma, M. Ahn, S. Levine, V. Kumar, K. Hausman, and S. Gu.  Emergent real-world robotic skills via unsupervised off-policy reinforcement learning.  _arXiv preprint arXiv:2004.12974_ , 2020.
  * Shen et al. [2020] Y. Shen, D. Guo, F. Long, L. A. Mateos, H. Ding, Z. Xiu, R. B. Hellman, A. King, S. Chen, C. Zhang, et al.  Robots under covid-19 pandemic: A comprehensive survey.  _Ieee Access_ , 9:1590–1615, 2020.
  * Shin et al. [2022] Y.-H. Shin, S. Hong, S. Woo, J. Choe, H. Son, G. Kim, J.-H. Kim, K. Lee, J. Hwangbo, and H.-W. Park.  Design of kaist hound, a quadruped robot platform for fast and efficient locomotion with mixed-integer nonlinear optimization of a gear train.  In _2022 International Conference on Robotics and Automation (ICRA)_ , pages 6614–6620. IEEE, 2022.
  * Siekmann et al. [2021] J. Siekmann, Y. Godse, A. Fern, and J. Hurst.  Sim-to-real learning of all common bipedal gaits via periodic reward composition.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 7309–7315. IEEE, 2021.
  * Sleiman et al. [2021] J.-P. Sleiman, F. Farshidian, M. V. Minniti, and M. Hutter.  A unified mpc framework for whole-body dynamic locomotion and manipulation.  _IEEE Robotics and Automation Letters_ , 6(3):4688–4695, 2021.
  * Smith [2016] F. Smith.  _Sarcos_ , pages 1–11.  Springer Netherlands, Dordrecht, 2016.  ISBN 978-94-007-7194-9.  10.1007/978-94-007-7194-9_22-1.  URL https://doi.org/10.1007/978-94-007-7194-9_22-1.
  * Smith et al. [2022] L. Smith, I. Kostrikov, and S. Levine.  A walk in the park: Learning to walk in 20 minutes with model-free reinforcement learning.  _arXiv preprint arXiv:2208.07860_ , 2022.
  * Smith et al. [2023] L. Smith, J. C. Kew, T. Li, L. Luu, X. B. Peng, S. Ha, J. Tan, and S. Levine.  Learning and adapting agile locomotion skills by transferring experience.  _arXiv preprint arXiv:2304.09834_ , 2023.
  * Smith et al. [2005] R. Smith et al.  Open dynamics engine, 2005.  URL https://www.ode.org/.
  * Sorokin et al. [2022] M. Sorokin, J. Tan, C. K. Liu, and S. Ha.  Learning to navigate sidewalks in outdoor environments.  _IEEE Robotics and Automation Letters_ , 7(2):3906–3913, 2022.
  * Spröwitz et al. [2013] A. Spröwitz, A. Tuleu, M. Vespignani, M. Ajallooeian, E. Badri, and A. J. Ijspeert.  Towards dynamic trot gait locomotion: Design, control, and experiments with cheetah-cub, a compliant quadruped robot.  _The International Journal of Robotics Research_ , 32(8):932–950, 2013.
  * Suh et al. [2022] H. J. T. Suh, T. Pang, and R. Tedrake.  Bundled gradients through contact via randomized smoothing.  _IEEE Robotics and Automation Letters_ , 7(2):4000–4007, 2022.
  * Sutton and Barto [2018] R. S. Sutton and A. G. Barto.  _Reinforcement learning: An introduction_.  MIT press, 2018.
  * Tan et al. [2018] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke.  Sim-to-real: Learning agile locomotion for quadruped robots.  _arXiv preprint arXiv:1804.10332_ , 2018.
  * Tang et al. [2023] Y. Tang, W. Yu, J. Tan, H. Zen, A. Faust, and T. Harada.  Saytap: Language to quadrupedal locomotion.  _arXiv preprint arXiv:2306.07580_ , 2023.
  * Taouil et al. [2023] I. Taouil, G. Turrisi, D. Schleich, V. Barasuol, C. Semini, and S. Behnke.  Quadrupedal footstep planning using learned motion models of a black-box controller.  In _2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 800–806. IEEE, 2023.
  * Tassa and Todorov [2011] Y. Tassa and E. Todorov.  Stochastic complementarity for local control of discontinuous dynamics.  In _Proceedings of Robotics: Science and Systems_ , 2011.
  * Tassa et al. [2012] Y. Tassa, T. Erez, and E. Todorov.  Synthesis and stabilization of complex behaviors through online trajectory optimization.  In _2012 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 4906–4913. IEEE, 2012.
  * Team et al. [2023] G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al.  Gemini: a family of highly capable multimodal models.  _arXiv preprint arXiv:2312.11805_ , 2023.
  * Tedrake et al. [2005] R. Tedrake, T. W. Zhang, H. S. Seung, et al.  Learning to walk in 20 minutes.  In _Proceedings of the Fourteenth Yale Workshop on Adaptive and Learning Systems_ , volume 95585, pages 1939–1412. Beijing, 2005.
  * Thananjeyan et al. [2021] B. Thananjeyan, A. Balakrishna, S. Nair, M. Luo, K. Srinivasan, M. Hwang, J. E. Gonzalez, J. Ibarz, C. Finn, and K. Goldberg.  Recovery rl: Safe reinforcement learning with learned recovery zones.  _IEEE Robotics and Automation Letters_ , 6(3):4915–4922, 2021.
  * Theodorou et al. [2010] E. Theodorou, J. Buchli, and S. Schaal.  Reinforcement learning of motor skills in high dimensions: A path integral approach.  In _2010 IEEE International Conference on Robotics and Automation_ , pages 2397–2403. IEEE, 2010.
  * Tobin et al. [2017] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel.  Domain randomization for transferring deep neural networks from simulation to the real world.  In _2017 IEEE/RSJ international conference on intelligent robots and systems (IROS)_ , pages 23–30. IEEE, 2017.
  * Todorov et al. [2012] E. Todorov, T. Erez, and Y. Tassa.  Mujoco: A physics engine for model-based control.  In _2012 IEEE/RSJ international conference on intelligent robots and systems_ , pages 5026–5033. IEEE, 2012.
  * Touvron et al. [2023] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023.
  * Truong et al. [2023] J. Truong, M. Rudolph, N. H. Yokoyama, S. Chernova, D. Batra, and A. Rai.  Rethinking sim2real: Lower fidelity simulation leads to higher sim2real transfer in navigation.  In _Conference on Robot Learning_ , pages 859–870. PMLR, 2023.
  * Tsounis et al. [2020] V. Tsounis, M. Alge, J. Lee, F. Farshidian, and M. Hutter.  Deepgait: Planning and control of quadrupedal gaits using deep reinforcement learning.  _IEEE Robotics and Automation Letters_ , 5(2):3699–3706, 2020.
  * Unitree [2021] Unitree.  Unitree legged robots, 2021.  URL https://www.unitree.com.
  * Van Den Oord et al. [2017] A. Van Den Oord, O. Vinyals, et al.  Neural discrete representation learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Van Hasselt et al. [2016] H. Van Hasselt, A. Guez, and D. Silver.  Deep reinforcement learning with double q-learning.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 30, 2016.
  * Viereck and Righetti [2021] J. Viereck and L. Righetti.  Learning a centroidal motion planner for legged locomotion.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 4905–4911. IEEE, 2021.
  * Viereck et al. [2022] J. Viereck, A. Meduri, and L. Righetti.  Valuenetqp: Learned one-step optimal control for legged locomotion.  In _Learning for Dynamics and Control Conference_ , pages 931–942. PMLR, 2022.
  * Villarreal et al. [2020] O. Villarreal, V. Barasuol, P. M. Wensing, D. G. Caldwell, and C. Semini.  Mpc-based controller with terrain insight for dynamic legged locomotion.  In _2020 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 2436–2442. IEEE, 2020.
  * Vollenweider et al. [2023] E. Vollenweider, M. Bjelonic, V. Klemm, N. Rudin, J. Lee, and M. Hutter.  Advanced skills through multiple adversarial motion priors in reinforcement learning.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 5120–5126. IEEE, 2023.
  * Wabersich et al. [2023] K. P. Wabersich, A. J. Taylor, J. J. Choi, K. Sreenath, C. J. Tomlin, A. D. Ames, and M. N. Zeilinger.  Data-driven safety filters: Hamilton-jacobi reachability, control barrier functions, and predictive methods for uncertain systems.  _IEEE Control Systems Magazine_ , 43(5):137–177, 2023.
  * Wang et al. [2022] J. Wang, T. S. Lembono, S. Kim, S. Calinon, S. Vijayakumar, and S. Tonneau.  Learning to guide online multi-contact receding horizon planning.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 12942–12949. IEEE, 2022.
  * Wang et al. [2016] Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas.  Dueling network architectures for deep reinforcement learning.  In _International conference on machine learning_ , pages 1995–2003. PMLR, 2016.
  * Watkins and Dayan [1992] C. J. Watkins and P. Dayan.  Q-learning.  _Machine learning_ , 8:279–292, 1992.
  * Wensing et al. [2017] P. M. Wensing, A. Wang, S. Seok, D. Otten, J. Lang, and S. Kim.  Proprioceptive actuator design in the mit cheetah: Impact mitigation and high-bandwidth physical interaction for dynamic legged robots.  _Ieee transactions on robotics_ , 33(3):509–522, 2017.
  * Wensing et al. [2023] P. M. Wensing, M. Posa, Y. Hu, A. Escande, N. Mansard, and A. Del Prete.  Optimization-based control for dynamic legged robots.  _IEEE Transactions on Robotics_ , 2023.
  * Werling et al. [2021] K. Werling, D. Omens, J. Lee, I. Exarchos, and C. K. Liu.  Fast and feature-complete differentiable physics for articulated rigid bodies with contact.  _arXiv preprint arXiv:2103.16021_ , 2021.
  * Widmer et al. [2023] D. Widmer, D. Kang, B. Sukhija, J. Hübotter, A. Krause, and S. Coros.  Tuning legged locomotion controllers via safe bayesian optimization.  In _Conference on Robot Learning_ , pages 2444–2464. PMLR, 2023.
  * Wieber [2006] P.-B. Wieber.  Trajectory free linear model predictive control for stable walking in the presence of strong perturbations.  In _2006 6th IEEE-RAS International Conference on Humanoid Robots_ , pages 137–142. IEEE, 2006.
  * Wijmans et al. [2019] E. Wijmans, A. Kadian, A. Morcos, S. Lee, I. Essa, D. Parikh, M. Savva, and D. Batra.  Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames.  _arXiv preprint arXiv:1911.00357_ , 2019.
  * Winkler et al. [2018] A. W. Winkler, C. D. Bellicoso, M. Hutter, and J. Buchli.  Gait and trajectory optimization for legged systems through phase-based end-effector parameterization.  _IEEE Robotics and Automation Letters_ , 3(3):1560–1567, 2018.
  * Won et al. [2022] J. Won, D. Gopinath, and J. Hodgins.  Physics-based character controllers using conditional vaes.  _ACM Transactions on Graphics (TOG)_ , 41(4):1–12, 2022.
  * Wu et al. [2023a] J. Wu, G. Xin, C. Qi, and Y. Xue.  Learning robust and agile legged locomotion using adversarial motion priors.  _IEEE Robotics and Automation Letters_ , 2023a.
  * Wu et al. [2023b] P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg.  Daydreamer: World models for physical robot learning.  In _Conference on Robot Learning_ , pages 2226–2240. PMLR, 2023b.
  * Xiao et al. [2023] X. Xiao, J. Liu, Z. Wang, Y. Zhou, Y. Qi, Q. Cheng, B. He, and S. Jiang.  Robot learning in the era of foundation models: A survey.  _arXiv preprint arXiv:2311.14379_ , 2023.
  * Xie et al. [2020a] Z. Xie, P. Clary, J. Dao, P. Morais, J. Hurst, and M. Panne.  Learning locomotion skills for cassie: Iterative design and sim-to-real.  In _Conference on Robot Learning_ , pages 317–329. PMLR, 2020a.
  * Xie et al. [2020b] Z. Xie, H. Y. Ling, N. H. Kim, and M. van de Panne.  Allsteps: curriculum-driven learning of stepping stone skills.  In _Computer Graphics Forum_ , volume 39, pages 213–224. Wiley Online Library, 2020b.
  * Xie et al. [2021] Z. Xie, X. Da, M. Van de Panne, B. Babich, and A. Garg.  Dynamics randomization revisited: A case study for quadrupedal locomotion.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 4955–4961. IEEE, 2021.
  * Xie et al. [2022] Z. Xie, X. Da, B. Babich, A. Garg, and M. v. de Panne.  Glide: Generalizable quadrupedal locomotion in diverse environments with a centroidal model.  In _International Workshop on the Algorithmic Foundations of Robotics_ , pages 523–539. Springer, 2022.
  * Xu et al. [2023] M. Xu, P. Huang, W. Yu, S. Liu, X. Zhang, Y. Niu, T. Zhang, F. Xia, J. Tan, and D. Zhao.  Creative robot tool use with large language models.  _arXiv preprint arXiv:2310.13065_ , 2023.
  * Xu et al. [2022] S. Xu, L. Zhu, and C. P. Ho.  Learning efficient and robust multi-modal quadruped locomotion: A hierarchical approach.  In _2022 international conference on robotics and automation (ICRA)_ , pages 4649–4655. IEEE, 2022.
  * Xu et al. [2024] Z. Xu, A. H. Raj, X. Xiao, and P. Stone.  Dexterous legged locomotion in confined 3d spaces with reinforcement learning.  _arXiv preprint arXiv:2403.03848_ , 2024.
  * Yam et al. [2023] K. C. Yam, P. M. Tang, J. C. Jackson, R. Su, and K. Gray.  The rise of robots increases job insecurity and maladaptive workplace behaviors: Multimethod evidence.  _Journal of Applied Psychology_ , 108(5):850, 2023\.
  * Yamane and Nakamura [2006] K. Yamane and Y. Nakamura.  Stable penalty-based model of frictional contacts.  In _Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006._ , pages 1904–1909. IEEE, 2006.
  * Yang et al. [2022a] L. Yang, Z. Li, J. Zeng, and K. Sreenath.  Bayesian optimization meets hybrid zero dynamics: Safe parameter learning for bipedal locomotion control.  In _2022 International Conference on Robotics and Automation (ICRA)_ , pages 10456–10462. IEEE, 2022a.
  * Yang et al. [2021] R. Yang, M. Zhang, N. Hansen, H. Xu, and X. Wang.  Learning vision-guided quadrupedal locomotion end-to-end with cross-modal transformers.  _arXiv preprint arXiv:2107.03996_ , 2021.
  * Yang et al. [2023a] R. Yang, Z. Chen, J. Ma, C. Zheng, Y. Chen, Q. Nguyen, and X. Wang.  Generalized animal imitator: Agile locomotion with versatile motion prior.  _arXiv preprint arXiv:2310.01408_ , 2023a.
  * Yang et al. [2023b] R. Yang, G. Yang, and X. Wang.  Neural volumetric memory for visual locomotion control.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1430–1440, 2023b.
  * Yang et al. [2022b] T.-Y. Yang, T. Zhang, L. Luu, S. Ha, J. Tan, and W. Yu.  Safe reinforcement learning for legged locomotion.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 2454–2461. IEEE, 2022b.
  * Yang et al. [2020] Y. Yang, K. Caluwaerts, A. Iscen, T. Zhang, J. Tan, and V. Sindhwani.  Data efficient reinforcement learning for legged robots.  In _Conference on Robot Learning_ , pages 1–10. PMLR, 2020.
  * Yang et al. [2022c] Y. Yang, T. Zhang, E. Coumans, J. Tan, and B. Boots.  Fast and efficient locomotion via learned gait transitions.  In _Conference on robot learning_ , pages 773–783. PMLR, 2022c.
  * Yang et al. [2023c] Y. Yang, X. Meng, W. Yu, T. Zhang, J. Tan, and B. Boots.  Learning semantics-aware locomotion skills from human demonstration.  In _Conference on robot learning_ , pages 2205–2214. PMLR, 2023c.
  * Yeganegi et al. [2019] M. H. Yeganegi, M. Khadiv, S. A. A. Moosavian, J.-J. Zhu, A. Del Prete, and L. Righetti.  Robust humanoid locomotion using trajectory optimization and sample-efficient learning.  In _2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids)_ , pages 170–177. IEEE, 2019.
  * Yeganegi et al. [2021] M. H. Yeganegi, M. Khadiv, A. Del Prete, S. A. A. Moosavian, and L. Righetti.  Robust walking based on mpc with viability guarantees.  _IEEE Transactions on Robotics_ , 2021.
  * Youm et al. [2023] D. Youm, H. Jung, H. Kim, J. Hwangbo, H.-W. Park, and S. Ha.  Imitating and finetuning model predictive control for robust and symmetric quadrupedal locomotion.  _IEEE Robotics and Automation Letters_ , 2023.
  * Yu et al. [2017] W. Yu, J. Tan, C. K. Liu, and G. Turk.  Preparing for the unknown: Learning a universal policy with online system identification.  In _Proceedings of Robotics: Science and Systems_ , Cambridge, Massachusetts, July 2017.  10.15607/RSS.2017.XIII.048.
  * Yu et al. [2019] W. Yu, V. C. Kumar, G. Turk, and C. K. Liu.  Sim-to-real transfer for biped locomotion.  In _2019 ieee/rsj international conference on intelligent robots and systems (iros)_ , pages 3503–3510. IEEE, 2019.
  * Yu et al. [2020] W. Yu, J. Tan, Y. Bai, E. Coumans, and S. Ha.  Learning fast adaptation with meta strategy optimization.  _IEEE Robotics and Automation Letters_ , 5(2):2950–2957, 2020.
  * Yu et al. [2021] W. Yu, D. Jain, A. Escontrela, A. Iscen, P. Xu, E. Coumans, S. Ha, J. Tan, and T. Zhang.  Visual-locomotion: Learning to walk on complex terrains with vision.  In _5th Annual Conference on Robot Learning_ , 2021.
  * Yu et al. [2023a] W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al.  Language to rewards for robotic skill synthesis.  In _Conference on Robot Learning_ , pages 374–404. PMLR, 2023a.
  * Yu et al. [2023b] W. Yu, C. Yang, C. McGreavy, E. Triantafyllidis, G. Bellegarda, M. Shafiee, A. J. Ijspeert, and Z. Li.  Identifying important sensory feedback for learning locomotion skills.  _Nature Machine Intelligence_ , 5(8):919–932, 2023b.
  * Yuan et al. [2019] K. Yuan, I. Chatzinikolaidis, and Z. Li.  Bayesian optimization for whole-body control of high-degree-of-freedom robots through reduction of dimensionality.  _IEEE Robotics and Automation Letters_ , 4(3):2268–2275, 2019.
  * Zhang et al. [2023] C. Zhang, N. Rudin, D. Hoeller, and M. Hutter.  Learning agile locomotion on risky terrains.  _arXiv preprint arXiv:2311.10484_ , 2023.
  * Zhang et al. [2022] H. Zhang, L. He, and D. Wang.  Deep reinforcement learning for real-world quadrupedal locomotion: a comprehensive review.  _Intelligence and Robotics_ , 2(3), 2022.
  * Zhou et al. [2023] H. Zhou, X. Yao, Y. Meng, S. Sun, Z. BIng, K. Huang, and A. Knoll.  Language-conditioned learning for robotic manipulation: A survey.  _arXiv preprint arXiv:2312.10807_ , 2023.
  * Zhu et al. [2023] Q. Zhu, H. Zhang, M. Lan, and L. Han.  Neural categorical priors for physics-based character control.  _ACM Transactions on Graphics (TOG)_ , 42(6):1–16, 2023.
  * Zhu [2023] T. Zhu.  _Design of a highly dynamic humanoid robot_.  University of California, Los Angeles, 2023.
  * Zhuang et al. [2023] Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao.  Robot parkour learning.  _arXiv preprint arXiv:2309.05665_ , 2023.


# Color-based classification of EEG Signals for people with severe locomotive
disorder

Shrestha, Ankit<EMAIL_ADDRESS> Adhikari, Bikram<EMAIL_ADDRESS>
###### Abstract

The neurons in the brain produces electric signals and a collective firing of
these electric signals gives rise to brainwaves. These brainwave signals are
captured using EEG (Electroencephalogram) devices as micro voltages. These
sequence of signals captured by EEG sensors have embedded features in them
that can be used for classification. The signals can be used as an alternative
input for people suffering from severe locomotive disorder.Classification of
different colors can be mapped for many functions like directional movement.
In this paper, raw EEG signals from NeuroSky Mindwave headset (a single
electrode EEG sensor) have been classified with an attention based Deep
Learning Network. Attention based LSTM Networks have been implemented for
classification of two different colors and four different colors. An accuracy
of 93.5% was obtained for classification of two colors and an accuracy of
65.75% was obtained for classifcation of four signals using the mentioned
attention based LSTM network.

###### Index Terms:

Electroencephalogram, Deep Neural Networks, NeuroSky, EEG-Raw signals, LSTM,
Attention layer, Raspberry Pi, Arduino

##  I Introduction

According to WHO, 15% (around 1 billion) people of total world population
lives with various form of disability. Among which 2-4% are suffering from
severe form of disability [1].

Many researches are carried out to improve the quality of life of people with
disabilities. Among many, Brain Computer Interface (BCI) is one of the highly
researched. BCI is a technology where a person can control the external device
directly through thought process. Electroencephalography (EEG) is one of the
widely used method to track brain wave patterns for BCI. Various devices are
developed to help disabled people in day to day activity like smart wheelchair
[2], communication system for people with lock-in syndrome [3], and using EEG
and BCI for communication.[4]

As of 2018, with a consumer toll of 3.3 million wheelchair users alone in USA,
of which 1.825 of the age 65 and above,[5] there has been a lot of research
with making a wheelchair more accessible and easy to use and a lot of the
research was focused on finding out ways for people with wheelchair to use it
without any assistance.

With EEG being one of the areas researched upon, this research will be mainly
focused on the application of BCI and EEG signal classification to improve the
quality of life of people with disabilities. This paper explains how BCI and
EEG in sync help affected person to control locomotion and communication on
their own.

##  II Literature review

BCI-EEG has become a huge field of research in the present days in order to
augment the brain through computer. A wide range of BCI-EEG application have
been researched and developed such as mind controlled robots, smart
environment control, etc.

A BCI for controlling a lower limb exoskeleton was developed by the scientists
from Korea University and TU Berlin[6] with the help of EEG cap to control
forward, left and right movement as well as to stand and sit by simply staring
at flickering light emitting diodes (LEDs). This application used steady-state
visual potential (SSVEP) approach along with canonical correlation analysis
(CCA) method in combination with k-nearest neighbors.

Another approach to BCI was to control Smart Home Environment using Conceptual
Imagery [7], where the semantics of the interaction is compatible with the
semantic of the task. For e.g., when the user imagine the concept of a TV set
then BCI recognizes the concept and generates a command to turn on the TV set
for the user to watch. This concept has been experimented by Nataliya Kosmyna
and her team with the help of “Domus”, a fully functional 40 meter square
smart home with a bedroom, a kitchen, a bathroom and a living room. They used
an asynchronous BCI system based on a Minimum Distance Classifier (MDCs) that
produced an accuracy in task of 77% in healthy subjects and a better accuracy
of 81% in disabled subjects.

Puja Sengupta proposed a Continuous Monitoring System [8], for analyzing the
Locked-In Syndrome(LIS) Patients and Comatose Patients to help them be
diagnosed properly by continuously monitoring the patient. The system used
Emotiv EpocPlus Headset to collect the EEG signals of the Coma patient and the
signals were classified using a Multilayer Feedforward Neural Network
including Restricted Boltzmann Machine(RBM) algorithm in the first hidden
layer and Sigmoid function and Radial Basis Function(RBF) algorithm in the
second hidden layer.

Intelligent adaptive user interface (iAUI) was used in a EEG Based Mobile
Remote Control for Wheel Chair Navigation through adaptive brain-robot
interface [9].

Another approach may be through intent recognition [10], which can be used for
smart applications such as turning off the led or controlling a robot. LSTM
[hochreiter1997long] tuned using implementation of Orthogonal Array
experiment, they were able to achieve considerably higher accuracy.

This paper describes the implementation of BCI in the medical field to help
people with disabilities by researching on the development of an application
that is capable of mind-controlled wheelchair movement and detection of coma
patients’ thoughts. Neurosky Mindwave has been used for the collection of EEG
data and Deep learning approach has been used for the classification of the
data obtained. With the implementation of Recurrent Neural Network (RNN) and
Long Short Term Memory (LSTM) neural network along with attention layer, there
has been a significant improvement in the results obtained in comparison to
the previous approaches used.

##  III Methodology

###  III-A Data Acquisition and Pre-processing

Neurosky Mindwave, a single electrode EEG headset, has been used for the
purpose of data collection for this project. The collection of data was
carried out on unstressed and natural conditions with minimal interference and
noise from the environment.

The data was collected at 512 Hz which is the maximum frequency that this EEG
headset supports. Four different colors were used as stimuli for the input
signal to EEG: red, blue, black and yellow. A corresponding label/command of
0, 1, 2 and 3 were mapped with the colors, respectively.

The subjects were instructed to first input the particular color that they are
going to think of and then focus on their thoughts for the next four seconds.
The headset captured the data for 4 seconds resulting in a sequence of 2048
numbers. The data with its attention level and signal quality were then stored
along with the input label.

The details are given in table I.

Table I: Command for signals Action |  Color |  Command
---|---|---
left |  red |  0
right |  blue |  1
up |  black |  2
down |  yellow |  3

During data collection, the obtained signals were filtered for outliers and
abnormal/error signals to minimize noise in time of input. The data was then
introduced to shape changes for feeding to neural network. The shape was
changed from a long sequence of 2048 to a shape of (256, 8). Further
preprocessing was not done and the raw data was used for classification in the
mentioned state.

###  III-B Classification Model

Deep learning approaches were applied for the classification of the EEG
signals. Recurrent Neural Networks (RNNs) were used in the neural network
architecture as the RNNs can preserve the dynamic temporal behavior of the
sequences. Many variations and types of RNNs were tried to realize the deep
neural net for classification. After many trials, LSTM network with an
attention layer was chosen as the best fit for this problem.

An implementation of LSTM neural network with attention layer was carried out
to classify the signals. Two LSTM layers with 256 memory units and 64 memory
units respectively were stacked for this neural network-based classifier. The
first LSTM layer was designed to input sequences of shape (512,4) and was then
connected to the dropout layer which was activated using Leaky-RELU. This was
connected to the second LSTM layer with 64 memory units which was connected to
an attention layer that learnt to focus on the important features of the
signal. The output of the attention layer was then fed to the dense layer with
SoftMax activation which acted as the output layer for the classifier. Adam
algorithm, an extension of the stochastic gradient descent (SGD), was used as
the optimizer for the classifier. Cross Entropy was used as the loss function
for the classifier. Since, this was a multiclass problem, Categorical Cross
Entropy was used as the loss function.

The same model of the neural network was trained using different class of
data. A four-class classifier was implemented for applications like Direction
control of Wheelchair and a two-class classifier was implemented for
applications like Detection of Coma patient thought for Yes/No questions. For
the two-class classifier the two colors with the best accuracy was used as
input after multiple trials.

A brief introduction to the network models are given in the following
sections.

####  III-B1 RNN

A recurrent neural network (RNN) is a class of artificial neural network where
connections between nodes form a directed graph along a sequence. This allows
it to exhibit dynamic temporal behavior for a time sequence. Unlike feed-
forward neural networks, RNNs can use their internal state (memory) to process
sequences of inputs. [11]

####  III-B2 Bi-directional RNN

Bi-directional RNNs use a finite sequence to predict or label each element of
the sequence based on the element’s past and future contexts. This is done by
concatenating the outputs of two RNNs, one processing the sequence from left
to right, the other one from right to left. The combined outputs are the
predictions of the teacher-given target signals. This technique has proved to
be especially useful when combined with LSTM RNNs.

Figure 1: The actual prototype for the wheelchair implemented for the research
Figure 2: The yes/no question response simulator prototype being tested among
test subjects

####  III-B3 Long short-term memory (LSTM)

Long short-term memory (LSTM) [12] is a deep learning system that avoids the
vanishing gradient problem. LSTM is normally augmented by recurrent gates
called “forget” gates. LSTM prevents back propagated errors from vanishing or
exploding. Instead, errors can flow backwards through unlimited numbers of
virtual layers unfolded in space. That is, LSTM can learn tasks that require
memories of events that happened thousands or even millions of discrete time
steps earlier.

####  III-B4 Attention Network

An attention function can be described as mapping a query and a set of key-
value pairs to an output, where the query, keys, values, and output are all
vectors. [13] The output is computed as a weighted sum of the values, where
the weight assigned to each value is computed by a compatibility function of
the query with the corresponding key

Now for the wheelchair, a simple raspberry 3.0 powered wheelchair with two
motors for two side wheel was used to manure the wheelchair. The table I.
demonstrates the motor state as per the signal received:

Table II: Command for Wheelchair and the motor state Signal |  Motor1 |  Motor2 |  Action
---|---|---|---
0 |  Anti-clockwise |  Anti-clockwise |  Back
1 |  Clockwise |  Clockwise |  Front
2 |  Off |  Clockwise |  Left
3 |  Clockwise |  Off |  Right

and the figure: 1 refrences the prototype model being implemented for the four
channel classifier.

Figure 3: The wireframe design for the two motor based programmable car chasis
used as a wheelchair prototype.

##  IV Result and Analysis

###  IV-A Specification of System

Keras framework with tensorflow was used to build this model. It was trained
on 1050 TI 4 GB memory, I5 8th gen 8300H with 8 GB ram. For the wheelchair
prototype, an arduino uno, raspberry pi 3.0 and a programmable car chasis set
with two motors for left and right wheels were used.

###  IV-B Approach

The system was trained using the EEG signals for four colors (red-blue-black-
yellow). The model was trained using a complete data set, with a validation
split to obtain the point after which the data starts over-fitting in the
network. After this, the training data at hand were again fed to train the
neural network model upto the point where the model is most appropriately fit.
The EEG data set used in the test were recorded for four colors and contained
both trained and untrained data.

Of 200 EEG signals collected from numerous subjects that were used for
training the system, 50 seen and 50 unseen data were used to test for each
color. For a four class classifier, the accuracy obtained for each direction
were combined to give out the total accuracy, whereas for two class
classifier, each color with an adjacent color were trained and the combination
showing the best accuracy based on trial was used for further processing.

Figure 4: The graph shows the training and validation accuracy obtained
between all four colors over 100 epochs. Figure 5: The graph shows the
training and validation loss obtained between all four colors over 100 epochs.
Figure 6: The graph shows the training and validation accuracy obtained
between red and black over 100 epochs. Figure 7: The graph shows the training
and validation loss obtained between red and black over 100 epochs. Figure 8:
The graph shows the training and validation accuracy obtained between red and
blue over 100 epochs. Figure 9: The graph shows the training and validation
loss obtained between red and blue over 100 epochs. Figure 10: The graph shows
the training and validation accuracy obtained between red and yellow over 100
epochs. Figure 11: The graph shows the training and validation loss obtained
between red and yellow over 100 epochs. Figure 12: The graph shows the
training and validation accuracy obtained between black and blue over 100
epochs. Figure 13: The graph shows the training and validation loss obtained
between black and blue over 100 epochs. Figure 14: The graph shows the
training and validation accuracy obtained between blue and yellow over 100
epochs. Figure 15: The graph shows the training and validation loss obtained
between blue and yellow over 100 epochs. Figure 16: The graph shows the
training and validation accuracy obtained between black and yellow over 100
epochs. Figure 17: The graph shows the training and validation loss obtained
between black and yellow over 100 epochs.

###  IV-C Analysis

The attention network fit was around 50 epochs for the four-color classifier
as in the figure 4 and figure 5. From figure c and figure d it can be analyzed
that for two color classifier using red and yellow as the colors (the best
case scenario), the fit was around 63 epochs as seen in figure 10 and 11. For
other two color classifiers, starting with red and blue, the fit was around 25
epochs as seen in figure 6 and figure 7, for red and black it was around 27
epochs as seen in figure 8 and figure 9, for blue and black it was around 65
epochs as seen in figure 12 and figure 13, for blue and yellow it was around
50 epochs as seen in figure 14 and figure 15.

It can be seen in the table attached that using a four color classifier in the
neural network the accuracy obtained is 63.75% whereas while using a two color
classifier accuracy jumps to 93.5% for the same data-set in the same system.

Table III: Result using two-color classification in attention network between Red and Yellow Set |  Data Type |  Color |  Command |  Number of data |  Correctly Predicted |  Falsely predicted |  Accur acy
---|---|---|---|---|---|---|---
1 |  Seen |  Red |  0 |  50 |  46 |  4 |  92%
2 |  Seen |  Yellow |  3 |  50 |  45 |  5 |  90%
3 |  Unseen |  Red |  0 |  50 |  48 |  2 |  96%
4 |  Unseen |  Yellow |  3 |  50 |  48 |  2 |  96%
|  |  |  |  |  Average |  93.5% |

Now, while comparing the four-color versus two-color classifiers, it can be
seen that four color classifiers showed comparatively low accuracy of 63.75%
compared to 93.5% accuracy of the two color classifier. This is because to
classify four signals the base accuracy starts from 25% whereas for two
signals it’s 50%, there’s 50-50 chance that the signals will be classified
correctly in a system classifying two signals.

Table IV: Result using four-color classification in attention network Set |  Data Type |  Color |  Command |  Number of data |  Correctly Predicted |  Falsely predicted |  Accur acy
---|---|---|---|---|---|---|---
1 |  Seen |  Red |  0 |  50 |  39 |  11 |  78%
2 |  Seen |  Blue |  1 |  50 |  38 |  12 |  76%
3 |  Seen |  Black |  2 |  50 |  27 |  23 |  54%
4 |  Seen |  Yellow |  3 |  50 |  36 |  14 |  72%
5 |  Unseen |  Red |  0 |  50 |  32 |  18 |  64%
6 |  Unseen |  Blue |  1 |  50 |  34 |  16 |  68%
7 |  Unseen |  Black |  2 |  50 |  28 |  22 |  56%
8 |  Unseen |  Yellow |  3 |  50 |  29 |  21 |  58%
|  |  |  |  |  Average |  65.75% |

The four-color classifier was then used to feed bluetooth signal to motor-
controlled wheelchair prototype with each color associated with the direction
the wheelchair would move as seen in the figure 3. To prevent the movement of
wheelchair with the slightest deviation of mind, another parameter i.e
attention was used, with which the wheelchair moved only with strong EEG
signals specifying to a color.

##  V Conclusion and Future Enhancement

The Attention based LSTM network classified the two colors with an accuracy of
93.5% and four colors with an accuracy of 65.75%.These models are considerable
improvements over the random classification of 50% and 25% for two class
classification problem and four class classification problem respectively.
These models can be used to suit many functions like answering of question
with yes/no for certain coma patients, directional control of wheelchairs for
patients of locomotive disorders, etc., with further improvements in the
future. There is enough room for improvement of the models by using techniques
like -introducing a better preprocessing method of data for removal of noise,
using a more realistic environment and stimuli to reflect real world
scenarios, using a more sophisticated EEG sensor with multiple electrodes to
get better signals from various regions of the brain used for thought and so
on.

## References

  * [1] W. WHO, “World report on disability,” _Geneva: WHO_ , 2011.
  * [2] C. Mandel, T. Lüth, T. Laue, T. Röfer, A. Gräser, and B. Krieg-Brückner, “Navigating a smart wheelchair with a brain-computer interface interpreting steady-state visual evoked potentials,” in _2009 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , Oct 2009, pp. 1118–1125.
  * [3] K. Ullah, M. Ali, M. Rizwan, and M. Imran, “Low-cost single-channel eeg based communication system for people with lock-in syndrome,” in _2011 IEEE 14th International Multitopic Conference_ , Dec 2011, pp. 120–125.
  * [4] J. R. Wolpaw, “Chapter 64 brain-computer interfaces (bcis) for communication and control: a mini-review,” _Advances in Clinical Neurophysiology, Proceedings of the 27th International Congress of Clinical Neurophysiology, AAEM 50th Anniversary and 57th Annual Meeting of the ACNS Joint Meeting Supplements to Clinical Neurophysiology_ , p. 607–613, 2004.
  * [5] “Wheelchair facts, numbers and figures [infographic].” [Online]. Available: https://kdsmartchair.com/blogs/news/18706123-wheelchair-facts-numbers-and-figures-infographic
  * [6] N.-S. Kwak, K.-R. Müller, and S.-W. Lee, “A lower limb exoskeleton control system based on steady state visual evoked potentials,” _Journal of neural engineering_ , vol. 12, p. 056009, 08 2015.
  * [7] N. Kosmyna, F. Tarpin-Bernard, N. Bonnefond, and B. Rivet, “Feasibility of bci control in a realistic smart home environment,” _Frontiers in Human Neuroscience_ , vol. 10, p. 416, 08 2016.
  * [8] e. a. Moulton R, “Continuous automated monitoring of somatosensory evoked potentials in posttraumatic coma. - pubmed - ncbi,” 2019. [Online]. Available: https://www.ncbi.nlm.nih.gov/pubmed/2030515
  * [9] V. Gandhi, G. Prasad, D. Coyle, L. Behera, and T. M. McGinnity, “Eeg-based mobile robot control through an adaptive brain–robot interface,” _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ , vol. 44, no. 9, pp. 1278–1285, 2014.
  * [10] X. Zhang, L. Yao, C. Huang, Q. Z. Sheng, and X. Wang, “Intent recognition in smart living through deep recurrent neural networks,” in _International Conference on Neural Information Processing_. Springer, 2017, pp. 748–758.
  * [11] Wikipedia contributors, “Recurrent neural network Wikipedia, the free encyclopedia,” 2019, [Online; accessed 29-July-2019].
  * [12] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural Comput._ , vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
  * [13] 2019\. [Online]. Available: https://medium.com/towards-artificial-intelligence/attention-is-all-you-need-transformer-4c34aa78308f