* [90] P. Haffner, “Connectionist Speech Recognition with a Global MMI Algorithm,” in _Proc. Eurospeech_ , Berlin, Germany, Dec. 1993, pp. 1929–1932.
  * [91] R. E. Bellman, _Dynamic Programming_. Princeton, NJ: Princeton University Press, 1957.
  * [92] A. Viterbi, “Error Bounds for Convolutional Codes and an Asymptotically Optimal Decoding Algorithm,” _IEEE Transactions on Information Theory_ , vol. 13, pp. 260–269, 1967.
  * [93] H. Ney, “The Use of a One-Stage Dynamic Programming Algorithm for Connected Word Recognition,” _IEEE Transactions on Acoustics, Speech, and Signal Processing_ , vol. 32, no. 2, pp. 263–271, 1984.
  * [94] W. Zhou, W. Michel, R. Schlüter, and H. Ney, “Efficient Training of Neural Transducer for Speech Recognition,” Apr. 2022, arXiv:2204.10586. [Online]. Available: https://arxiv.org/abs/2204.10586
  * [95] A. Zeyer, R. Schlüter, and H. Ney, “Why does CTC Result in Peaky Behavior?” Preprint arXiv:2105.14849, May 2021, arXiv:2105.14849. [Online]. Available: http://arxiv.org/abs/2105.14849
  * [96] A. Laptev, S. Majumdar, and B. Ginsburg, “CTC Variations Through New WFST Topologies,” in _Proc. Interspeech_ , Incheon, Korea, sep 2022. [Online]. Available: https://doi.org/10.21437
  * [97] X. He, L. Deng, and W. Chou, “Discriminative Learning in Sequential Pattern Recognition – A Unifying Review for Optimization-Oriented Speech Recognition,” _IEEE Signal Processing Magazine_ , vol. 25, no. 5, pp. 14–36, 2008.
  * [98] M. Zeineldeen, A. Glushko, W. Michel, A. Zeyer, R. Schlüter, and H. Ney, “Investigating Methods to Improve Language Model Integration for Attention-Based Encoder-Decoder ASR Models,” in _Proc. Interspeech_ , Brno, Czechia, Aug. 2021, pp. 2856–2860.
  * [99] N.-P. Wynands, W. Michel, J. Rosendahl, R. Schlüter, and H. Ney, “Efficient Sequence Training of Attention Models using Approximative Recombination,” in _Proc. IEEE ICASSP_ , Singapore, May 2022, arXiv:2110.09245.
  * [100] Z. Yang, W. Zhou, R. Schlüter, and H. Ney, “Lattice-Free Sequence Discriminative Training for Phoneme-based Neural Transducers,” in _Proc. IEEE ICASSP_ , Rhodes, Greece, Oct. 2022, submitted, preprint: arXiv:NNNN.NNNNN.
  * [101] V. Valtchev, J. J. Odell, P. C. Woodland, and S. J. Young, “MMIE Training of Large Vocabulary Recognition Systems,” _Speech Communication_ , vol. 22, no. 4, pp. 303–314, 1997.
  * [102] D. Povey and P. Woodland, “Improved Discriminative Training Techniques for Large Vocabulary Continuous Speech Recognition,” in _Proc. IEEE ICASSP_ , Salt Lake City, UT, May 2001, pp. 45–48.
  * [103] R. Schlüter, W. Macherey, B. Müller, and H. Ney, “Comparison of Discriminative Training Criteria and Optimization Methods for Speech Recognition,” _Speech Communication_ , vol. 34, no. 3, pp. 287–310, May 2001, EURASIP Best Paper Award.
  * [104] B. Kingsbury, “Lattice-Based Optimization of Sequence Classification Criteria for Neural-Network Acoustic Modeling,” in _Proc. IEEE ICASSP_ , Taipei, Taiwan, Apr. 2009, pp. 3761–3764.
  * [105] G. Heigold, R. Schlüter, H. Ney, and S. Wiesler, “Discriminative Training for Automatic Speech Recognition: Modeling, Criteria, Optimization, Implementation, and Performance,” _IEEE Signal Processing Magazine_ , vol. 29, no. 6, pp. 58–69, Nov. 2012.
  * [106] W. Michel, R. Schlüter, and H. Ney, “Comparison of Lattice-Free and Lattice-Based Sequence Discriminative Training Criteria for LVCSR,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 1601–1605. [Online]. Available: http://arxiv.org/abs/1907.01409
  * [107] R. Prabhavalkar, T. N. Sainath, Y. Wu, P. Nguyen, Z. Chen, C.-C. Chiu, and A. Kannan, “Minimum Word Error Rate Training for Attention-Based Sequence-to-Sequence Models,” in _Proc. IEEE ICASSP_ , Calgary, Alberta, Canada, Apr. 2018, pp. 4839–4843.
  * [108] C. Weng, C. Yu, J. Cui, C. Zhang, and D. Yu, “Minimum Bayes Risk Training of RNN-Transducer for End-to-End Speech Recognition,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 966–970.
  * [109] M. K. Baskar, L. Burget, S. Watanabe, M. Karafiát, T. Hori, and J. H. Černockỳ, “Promising Accurate Prefix Boosting for Sequence-to-Sequence ASR,” in _Proc. IEEE ICASSP_. Brighton, UK: IEEE, May 2019, pp. 5646–5650.
  * [110] A. Tjandra, S. Sakti, and S. Nakamura, “Sequence-to-Sequence ASR Optimization via Reinforcement Learning,” in _Proc. IEEE ICASSP_. Calgary, Alberta, Canada: IEEE, Apr. 2018, pp. 5829–5833.
  * [111] S. Karita, A. Ogawa, M. Delcroix, and T. Nakatani, “Sequence Training of Encoder-Decoder Model Using Policy Gradient for End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_. Calgary, Alberta, Canada: IEEE, Apr. 2018, pp. 5839–5843.
  * [112] S. Sabour, W. Chan, and M. Norouzi, “Optimal Completion Distillation for Sequence Learning,” in _Proc. ICLR_ , New Orleans, LA, May 2019, arXiv:1810.01398.
  * [113] W. Michel, R. Schlüter, and H. Ney, “Early Stage LM Integration Using Local and Global Log-Linear Combination,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 3605–3609. [Online]. Available: http://https://arxiv.org/abs/2005.10049
  * [114] G. E. Hinton, S. Osindero, and Y.-W. Teh, “A Fast Learning Algorithm for Deep Belief Nets,” _Neural Computation_ , vol. 18, no. 7, pp. 1527–1554, Jul. 2006.
  * [115] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy Layer-Wise Training of Deep Networks,” in _Proc. NIPS_ , Barcelona, Spain, Dec. 2006, pp. 153–160.
  * [116] A. Zeyer, P. Doetsch, P. Voigtlaender, R. Schlüter, and H. Ney, “A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition,” in _Proc. IEEE ICASSP_ , New Orleans, LA, Mar. 2017, pp. 2462–2466.
  * [117] A. Zeyer, T. Alkhouli, and H. Ney, “RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition,” in _Proc. ACL_ , Melbourne, Australia, Jul. 2018, pp. 128–133.
  * [118] A. Zeyer, K. Irie, R. Schlüter, and H. Ney, “Improved Training of End-to-End Attention Models for Speech Recognition,” in _Proc. Interspeech_ , Hyderabad, India, Sep. 2018, pp. 7–11.
  * [119] A. Zeyer, A. Merboldt, R. Schlüter, and H. Ney, “A Comprehensive Analysis on Attention Models,” in _Proc. NIPS_ , Montreal, Canada, Dec. 2018.
  * [120] Y. Chung, C. Wu, C. Shen, H. Lee, and L. Lee, “Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder,” _CoRR_ , vol. abs/1603.00982, 2016. [Online]. Available: http://arxiv.org/abs/1603.00982
  * [121] Y.-C. Chen, S.-F. Huang, H.-y. Lee, Y.-H. Wang, and C.-H. Shen, “Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 27, no. 9, pp. 1481–1493, 2019\.
  * [122] S. Scanzio, P. Laface, L. Fissore, R. Gemello, and F. Mana, “On the Use of a Multilingual Neural Network Front-End,” in _Proc. Interspeech_ , Brisbane, Australia, Sep. 2008, pp. 2711–2714.
  * [123] Z. Tüske, J. Pinto, D. Willett, and R. Schlüter, “Investigation on Cross- and Multilingual MLP features under matched and mismatched acoustical conditions,” in _IEEE International Conference on Acoustics, Speech, and Signal Processing_ , Vancouver, Canada, May 2013, pp. 7349–7353.
  * [124] S. Zhou, S. Xu, and B. Xu, “Multilingual End-to-End Speech Recognition with a Single Transformer on Low-Resource Languages,” Jun. 2018, arXiv:1806.05059. [Online]. Available: https://arxiv.org/abs/1806.05059
  * [125] O. Adams, M. Wiesner, S. Watanabe, and D. Yarowsky, “Massively Multilingual Adversarial Speech Recognition,” Apr. 2019, arXiv:1904.02210. [Online]. Available: https://arxiv.org/abs/1904.02210
  * [126] W. Hou, Y. Dong, B. Zhuang, L. Yang, J. Shi, and T. Shinozaki, “Large-scale end-to-end multilingual speech recognition and language identification with multi-task learning,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 1037–1041.
  * [127] V. Pratap, A. Sriram, P. Tomasello, A. Hannun, V. Liptchinsky, G. Synnaeve, and R. Collobert, “Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters,” 2020, arXiv:2007.03001. [Online]. Available: https://arxiv.org/abs/2007.03001
  * [128] B. Li, R. Pang, T. N. Sainath, A. Gulati, Y. Zhang, J. Qin, P. Haghani, W. R. Huang, M. Ma, and J. Bai, “Scaling End-to-End Models for Large-Scale Multilingual ASR,” in _Proc. IEEE ASRU_ , 2021, pp. 1011–1018.
  * [129] Y. Zhang, D. S. Park, W. Han, J. Qin, A. Gulati, J. Shor, A. Jansen, Y. Xu, Y. Huang, S. Wang, Z. Zhou, B. Li, M. Ma, W. Chan, J. Yu, Y. Wang, L. Cao, K. C. Sim, B. Ramabhadran, T. N. Sainath, F. Beaufays, Z. Chen, Q. V. Le, C.-C. Chiu, R. Pang, and Y. Wu, “BigSSL: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 16, no. 6, pp. 1519–1532, oct 2022. [Online]. Available: https://arxiv.org/abs/2109.13226
  * [130] Z. Chen, Y. Zhang, A. Rosenberg, B. Ramabhadran, P. J. Moreno, A. Bapna, and H. Zen, “MAESTRO: Matched Speech Text Representations through Modality Matching,” in _Proc. Interspeech_ , Incheon, South Korea, Sep. 2022, arXiv:2204.03409. [Online]. Available: https://arxiv.org/abs/2204.03409
  * [131] A. Radford, J. W. Kim, C. McLeavey, P. Mishkin, T. Xu, G. Brockman, and I. Sutskever, “Introducing Whisper - Robust Speech Recognition via Large-Scale Weak Supervision,” Sep. 2022. [Online]. Available: https://openai.com/blog/whisper/
  * [132] T. P. Vogl, J. Mangis, A. Rigler, W. Zink, and D. Alkon, “Accelerating the Convergence of the Back-Propagation Method,” _Biological Cybernetics_ , vol. 59, no. 4, pp. 257–263, 1988.
  * [133] N. S. Keskar and G. Saon, “A Nonmonotone Learning Rate Strategy for SGD Training of Deep Neural Networks,” in _Proc. IEEE ICASSP_. Queensland, Australia: IEEE, Apr. 2015, pp. 4974–4978.
  * [134] S. Renals, N. Morgan, H. Bourlard, C. Wooters, and P. Kohn, “Connectionist Speech Recognition: Status and Prospects,” ICSI, 1991, Tech. Rep. TR-OI-070.
  * [135] D. Johnson, D. Ellis, C. Oei, C. Wooters, and P. Faerber, “QuickNet,” ICSI, Berkeley, 2004. [Online]. Available: http://www.icsi.berkeley.edu/Speech/qn.html
  * [136] A. Senior, G. Heigold, M. Ranzato, and K. Yang, “An Empirical Study of Learning Rates in Deep Neural Networks for Speech Recognition,” in _Proc. IEEE ICASSP_. Vancouver, BC, Canada: IEEE, May 2013, pp. 6724–6728.
  * [137] I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regularization,” in _Proc. ICLR_ , New Orleans, LA, May 2019, arXiv:1711.05101.
  * [138] S. L. Smith, P.-J. Kindermans, C. Ying, and Q. V. Le, “Don’t Decay the Learning Rate, Increase the Batch Size,” in _Proc. ICLR_ , New Orleans, LA, May 2018, arXiv:1711.00489.
  * [139] J. Howard and S. Ruder, “Universal Language Model Fine-Tuning for Text Classification,” in _Proc. ACL_ , Melbourne, Australia, Jun. 2018, pp. 328–339.
  * [140] M. Jaderberg, V. Dalibard, S. Osindero, W. M. Czarnecki, J. Donahue, A. Razavi, O. Vinyals, T. Green, I. Dunning, K. Simonyan, C. Fernando, and K. Kavukcuoglu, “Population Based Training of Neural Networks,” Nov. 2017, arXiv:1711.09846. [Online]. Available: https://arxiv.org/abs/1711.09846v1
  * [141] T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, “Meta-Learning in Neural Networks: A Survey,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. PP, pp. 1–20, 2021.
  * [142] J. L. Elman, “Learning and Development in Neural Networks: The Importance of Starting Small,” _Cognition_ , vol. 48, no. 1, pp. 71–99, 1993. [Online]. Available: https://www.sciencedirect.com/science/article/pii/0010027793900584
  * [143] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum Learning,” in _Proc. ICML_ , Montreal, Quebec, Canada, Jun. 2009, p. 41–48.
  * [144] D. Amodei, R. Anubhai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, J. Chen, M. Chrzanowski, A. Coates, G. Diamos, E. Elsen, J. Engel, L. Fan, C. Fougner, T. Han, A. Hannun, B. Jun, P. LeGresley, L. Lin, S. Narang, A. Ng, S. Ozair, R. Prenger, J. Raiman, S. Satheesh, D. Seetapun, S. Sengupta, Y. Wang, Z. Wang, C. Wang, B. Xiao, D. Yogatama, J. Zhan, and Z. Zhu, “Deep Speech 2: End-to-End Speech Recognition in English and Mandarin,” in _Proc. ICML_ , New York City, NY, Jun. 2016, pp. 173–182.
  * [145] Z. Tüske, G. Saon, K. Audhkhasi, and B. Kingsbury, “Single Headed Attention Based Sequence-to-Sequence Model for State-of-the-Art Results on Switchboard,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 551–555.
  * [146] W. Zhang, X. Chang, Y. Qian, and S. Watanabe, “Improving End-to-End Single-Channel Multi-Talker Speech Recognition,” _IEEE/ACM Trans. Audio, Speech, and Language Processing_ , vol. 28, pp. 1385–1394, 2020.
  * [147] B. Polyak, “Some Methods of Speeding up the Convergence of Iteration Methods,” _USSR Computational Mathematics and Mathematical Physics_ , vol. 4, no. 5, pp. 1–17, 1964. [Online]. Available: https://www.sciencedirect.com/science/article/pii/0041555364901375
  * [148] Y. Nesterov, “A method of solving a convex programming problem with convergence rate O($\frac{1}{k^{2}}$),” _Soviet Mathematics Doklady_ , vol. 27, pp. 372–376, 1983.
  * [149] I. Sutskever, J. Martens, G. Dahl, and G. Hinton, “On the Importance of Initialization and Momentum in Deep Learning,” in _Proc. ICML_ , Atlanta, GA, Jun. 2013, pp. 1139–1147.
  * [150] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” Apr. 2014, arXiv:2104.02387. [Online]. Available: https://arxiv.org/abs/1412.6980
  * [151] Z. Tüske, G. Saon, and B. Kingsbury, “On the Limit of English Conversational Speech Recognition,” in _Proc. Interspeech_ , Brno, Czechia, Sep. 2021, pp. 2062–2066.
  * [152] P. Nakkiran, G. Kaplun, Y. Bansal, T. Yang, B. Barak, and I. Sutskever, “Deep Double Descent: Where Bigger Models and More Data Hurt,” Dec. 2019, arXiv:1912.02292. [Online]. Available: https://arxiv.org/abs/1912.02292
  * [153] A. Krogh and J. Hertz, “A Simple Weight Decay Can Improve Generalization,” in _Neural Information Processing Systems (NIPS)_ , Denver, CO, Dec. 1991, pp. 950–957.
  * [154] A. F. Murray and P. J. Edwards, “Enhanced MLP Performance and Fault Tolerance Resulting from Synaptic Weight Noise during Training,” _IEEE Transactions on Neural Networks_ , vol. 5, no. 5, pp. 792–802, Sep. 1994.
  * [155] A. Graves, “Practical Variational Inference for Neural Networks,” _Advances in Neural Information Processing Systems_ , vol. 24, 2011.
  * [156] A. Neelakantan, L. Vilnis, Q. V. Le, I. Sutskever, L. Kaiser, K. Kurach, and J. Martens, “Adding Gradient Noise Improves Learning for Very Deep Networks,” Nov. 2015, arXiv:1511.06807. [Online]. Available: https://arxiv.org/abs/1511.06807
  * [157] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, “Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors,” Jul. 2012, arXiv:1207.0580. [Online]. Available: https://arxiv.org/abs/1207.0580
  * [158] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” in _Advances in Neural Information Processing Systems (NIPS)_ , vol. 25, Lake Tahoe, NV, Dec. 2012.
  * [159] Y. Gal and Z. Ghahramani, “Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,” in _Proc. ICML_ , New York City, NY, Jun. 2016, pp. 1050–1059.
  * [160] G. Huang, Y. Sun, Z. Liu, D. Sedra, and K. Q. Weinberger, “Deep Networks with Stochastic Depth,” in _European Conference on Computer Vision_ , Amsterdam, Netherlands, Oct. 2016, pp. 646–661.
  * [161] N.-Q. Pham, T.-S. Nguyen, J. Niehues, M. Müller, and A. Waibel, “Very Deep Self-Attention Networks for End-to-End Speech Recognition,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 66–70.
  * [162] J. Lee and S. Watanabe, “Intermediate Loss Regularization for CTC-Based Speech Recognition,” in _Proc. IEEE ICASSP_ , Toronto, Ontario, Canada, Jun. 2021, pp. 6224–6228.
  * [163] L. Wan, M. Zeiler, S. Zhang, Y. Le Cun, and R. Fergus, “Regularization of Neural Networks using DropConnect,” in _Proc. ICML_ , 2013, pp. 1058–1066.
  * [164] D. Krueger, T. Maharaj, J. Kramár, M. Pezeshki, N. Ballas, N. R. Ke, A. Goyal, Y. Bengio, A. Courville, and C. Pal, “Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations,” Jun. 2016, arXiv:1606.01305. [Online]. Available: https://arxiv.org/abs/1606.01305
  * [165] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” in _IEEE Conf. on Computer Vision and Pattern Recognition_ , Las Vegas, NV, Jun. 2016, pp. 2818–2826.
  * [166] S. Bengio, O. Vinyals, N. Jaitly, and N. Shazeer, “Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks,” _Proc. NIPS_ , vol. 28, Dec. 2015.
  * [167] T. Trinh, A. Dai, T. Luong, and Q. Le, “Learning Longer-Term Dependencies in RNNs with Auxiliary Losses,” in _Proc. ICML_ , Stockholm, Sweden, Jul. 2018, pp. 4965–4974.
  * [168] R. J. Williams and J. Peng, “An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories,” _IEEE Neural Computation_ , vol. 2, no. 4, pp. 490–501, 1990.
  * [169] S. Merity, N. S. Keskar, and R. Socher, “An Analysis of Neural Language Modeling at Multiple Scales,” Mar. 2018, arXiv:1803.08240. [Online]. Available: https://arxiv.org/abs/1803.08240
  * [170] L. Meng, J. Xu, X. Tan, J. Wang, T. Qin, and B. Xu, “MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition,” in _Proc. IEEE ICASSP_. Toronto, Ontario, Canada: IEEE, Jun. 2021, pp. 7008–7012.
  * [171] S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift,” in _Proc. ICML_ , Lille, France, Jul. 2015, pp. 448–456.
  * [172] N. Kanda, R. Takeda, and Y. Obuchi, “Elastic Spectral Distortion for Low Resource Speech Recognition with Deep Neural Networks,” in _Proc. IEEE ASRU_ , Olomouc, Czech Republic, Dec. 2013, pp. 309–314.
  * [173] T. Ko, V. Peddinti, D. Povey, and S. Khudanpur, “Audio Augmentation for Speech Recognition,” in _Proc. Interspeech_ , Dresden, Germany, Sep. 2015\.
  * [174] N. Jaitly and G. E. Hinton, “Vocal Tract Length Perturbation (VTLP) Improves Speech Recognition,” in _Proc. ICML_ , vol. 117, Jun. 2013, p. 21.
  * [175] G. Saon, Z. Tüske, K. Audhkhasi, and B. Kingsbury, “Sequence Noise Injected Training for End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_ , Brighton, England, May 2019, pp. 6261–6265.
  * [176] D. S. Park, Y. Zhang, C.-C. Chiu, Y. Chen, B. Li, W. Chan, Q. V. Le, and Y. Wu, “SpecAugment on Large Scale Datasets,” in _Proc. IEEE ICASSP_ , Brighton, UK, May 2019, pp. 6879–6883.
  * [177] C. Wang, Y. Wu, Y. Du, J. Li, S. Liu, L. Lu, S. Ren, G. Ye, S. Zhao, and M. Zhou, “Semantic Mask for Transformer Based End-to-End Speech Recognition,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 971–975.
  * [178] T. Hayashi, S. Watanabe, Y. Zhang, T. Toda, T. Hori, R. Astudillo, and K. Takeda, “Back-Translation-Style Data Augmentation for End-to-End ASR,” in _Proc. IEEE SLT_. Athens, Greece: IEEE, Dec. 2018, pp. 426–433.
  * [179] N. Rossenbach, M. Zeineldeen, B. Hilmes, R. Schlüter, and H. Ney, “Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures,” in _Proc. IEEE ASRU_ , Cartagena, Colombia, Dec. 2021, arXiv:2104.05379.
  * [180] A. Graves, “Connectionist Temporal Classification,” in _Supervised Sequence Labelling with Recurrent Neural Networks_. Heidelberg, Germany: Springer, 2012, ch. Connectionist Temporal Classification, pp. 61–93.
  * [181] Y. Higuchi, S. Watanabe, N. Chen, T. Ogawa, and T. Kobayashi, “Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 3655–3659.
  * [182] W. Chan, C. Saharia, G. Hinton, M. Norouzi, and N. Jaitly, “Imputer: Sequence Modelling via Imputation and Dynamic Programming,” in _Proc. ICML_. PMLR, Jul. 2020, pp. 1403–1413.
  * [183] Y. Fujita, S. Watanabe, M. Omachi, and X. Chang, “Insertion-Based Modeling for End-to-End Automatic Speech Recognition,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 3660–3664.
  * [184] L. Dong and B. Xu, “Cif: Continuous Integrate-and-Fire for End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 6079–6083.
  * [185] J. Nozaki and T. Komatsu, “Relaxing the Conditional Independence Assumption of CTC-Based ASR by Conditioning on Intermediate Predictions,” in _Proc. Interspeech_ , Brno, Czechia, Sep. 2021, pp. 3735–3739.
  * [186] Y. Higuchi, N. Chen, Y. Fujita, H. Inaguma, T. Komatsu, J. Lee, J. Nozaki, T. Wang, and S. Watanabe, “A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generation,” Oct. 2021, arXiv:2110.05249. [Online]. Available: https://arxiv.org/abs/2110.05249
  * [187] J. Chorowski and N. Jaitly, “Towards Better Decoding and Language Model Integration in Sequence to Sequence Models,” in _Proc. Interspeech_ , Stockhol, Sweden, Aug. 2017, pp. 523–527.
  * [188] W. Zhou, R. Schlüter, and H. Ney, “Robust Beam Search for Encoder-Decoder Attention Based Speech Recognition without Length Bias,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 1768–1772.
  * [189] P. Koehn and R. Knowles, “Six Challenges for Neural Machine Translation,” in _First Workshop on Neural Machine Translation_. Vancouver, BC, Canada: Association for Computational Linguistics, Aug. 2017, pp. 28–39.
  * [190] T. Hori, S. Watanabe, and J. Hershey, “Joint CTC Attention Decoding for End-to-End Speech Recognition,” in _Proc. ACL_ , Vancouver, BC, Canada, Jul. 2017, pp. 518–529.
  * [191] Z. Tu, Z. Lu, Y. Liu, X. Liu, and H. Li, “Modeling Coverage for Neural Machine Translation,” in _Proc. ACL_ , Berlin, Germany, May 2016, pp. 76–85.
  * [192] T. Hori, J. Cho, and S. Watanabe, “End-to-End Speech Recognition with Word-Based RNN Language Models,” in _Proc. IEEE SLT_. Athens, Greece: IEEE, Dec. 2018, pp. 389–396.
  * [193] T. Hori and A. Nakamura, _Speech Recognition Algorithms Using Weighted Finite-State Transducers_. San Rafael, CA: Morgan & Claypool Publishers, 2013.
  * [194] Y. Miao, M. Gowayyed, and F. Metze, “EESEN: End-to-End Speech Recognition Using Deep RNN Models and WFST-Based Decoding,” in _Proc. IEEE ASRU_ , Scottsdale, AZ, Dec. 2015, pp. 167–174.
  * [195] R. Haeb-Umbach and H. Ney, “Improvements in Beam Search for 10000-Word Continuous-Speech Recognition,” _IEEE Transactions on Speech and Audio Processing_ , vol. 2, no. 2, pp. 353–356, 1994.
  * [196] H. Ney and S. Ortmanns, “Progress in Dynamic Programming Search for LVCSR,” _Proceedings of the IEEE_ , vol. 88, no. 8, pp. 1224–1240, Aug. 2000. [Online]. Available: http://dx.doi.org/10.1109/5.880081
  * [197] T. Hori, Y. Kubo, and A. Nakamura, “Real-Time One-Pass Decoding with Recurrent Neural Network Language Model for Speech Recognition,” in _Proc. IEEE ICASSP_ , Florence, Italy, May 2014, pp. 6364–6368.
  * [198] E. Beck, W. Zhou, R. Schlüter, and H. Ney, “LSTM Language Models for LVCSR in First-Pass Decoding and Lattice-Rescoring,” Jul. 2019, arXiv:1907.01030. [Online]. Available: https://arxiv.org/abs/1907.01030
  * [199] G. Saon, Z. Tüske, and K. Audhkhasi, “Alignment-Length Synchronous Decoding for RNN Transducer,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 7804–7808.
  * [200] A. Y. Hannun, A. L. Maas, D. Jurafsky, and A. Y. Ng, “First-Pass Large Vocabulary Continuous Speech Recognition Using Bi-Directional Recurrent DNNs,” Dec. 2014, arXiv:1408.2873. [Online]. Available: https://arxiv.org/abs/1408.2873
  * [201] N. Moritz, T. Hori, and J. Le Roux, “Triggered Attention for End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_. Brighton, UK: IEEE, May 2019, pp. 5666–5670.
  * [202] N. Moritz, T. Hori, and J. Le, “Streaming Automatic Speech Recognition with the Transformer Model,” in _Proc. IEEE ICASSP_. Barcelona, Spain: IEEE, May 2020, pp. 6074–6078.
  * [203] M. Jain, K. Schubert, J. Mahadeokar, C.-F. Yeh, K. Kalgaonkar, A. Sriram, C. Fuegen, and M. L. Seltzer, “RNN-T for Latency Controlled ASR with Improved Beam Search,” Nov. 2019, arXiv:1911.01629. [Online]. Available: https://arxiv.org/abs/1911.01629
  * [204] C.-F. Yeh, J. Mahadeokar, K. Kalgaonkar, Y. Wang, D. Le, M. Jain, K. Schubert, C. Fuegen, and M. L. Seltzer, “Transformer-Transducer: End-to-Snd Speech Recognition with Self-Attention,” in _Proc. IEEE ICASSP_ , Brighton, UK, May 2019, pp. 7829–7833.
  * [205] L. Lu, C. Liu, J. Li, and Y. Gong, “Exploring Transformers for Large-Scale Speech Recognition,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 5041–5045.
  * [206] T. Wang, Y. Fujita, X. Chang, and S. Watanabe, “Streaming End-to-End ASR Based on Blockwise Non-Autoregressive Models,” in _Proc. Interspeech_ , Brno, Czechia, Sep. 2021, pp. 3755–3759.
  * [207] E. Tsunoo, Y. Kashiwagi, and S. Watanabe, “Streaming Transformer ASR with Blockwise Synchronous Beam Search,” in _Proc. IEEE SLT_. Shenzhen, China: IEEE, Jun. 2021, pp. 22–29.
  * [208] K. Hwang and W. Sung, “Character-level language modeling with hierarchical recurrent neural networks,” in _Proc. IEEE ICASSP_. New Orleans, LA: IEEE, Mar. 2017, pp. 5720–5724.
  * [209] T. Hori, S. Watanabe, Y. Zhang, and W. Chan, “Advances in Joint CTC-Attention Based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM,” in _Proc. Interspeech_ , Stockhol, Sweden, Aug. 2017, pp. 949–953.
  * [210] A. Kannan, Y. Wu, P. Nguyen, T. N. Sainath, Z. Chen, and R. Prabhavalkar, “An Analysis of Incorporating an External Language Model into a Sequence-to-Sequence Model,” in _Proc. IEEE ICASSP_ , Calgary, Alberta, Canada, Apr. 2018, pp. 1–5828.
  * [211] R. Collobert, C. Puhrsch, and G. Synnaeve, “Wav2Letter: An End-to-End Convnet-Based Speech Recognition System,” Sep. 2016, arXiv:1609.03193. [Online]. Available: https://arxiv.org/abs/1609.03193
  * [212] Y. He, T. N. Sainath, R. Prabhavalkar, I. McGraw, R. Alvarez, D. Zhao, D. Rybach, A. Kannan, Y. Wu, R. Pang, Q. Liang, D. Bhatia, Y. Shangguan, B. Li, G. Pundak, K. C. Sim, T. Bagby, S.-y. Chang, K. Rao, and A. Gruenstein, “Streaming End-to-End Speech Recognition for Mobile Devices,” in _Proc. IEEE ICASSP_ , Brighton, UK, May 2019, pp. 6381–6385.
  * [213] G. Saon, Z. Tüske, D. Bolanos, and B. Kingsbury, “Advancing RNN Transducer Technology for Speech Recognition,” in _Proc. IEEE ICASSP_. Toronto, Ontario, Canada: IEEE, Jun. 2021, pp. 5654–5658, arXiv:2103.09935. [Online]. Available: https://arxiv.org/abs/2103.09935
  * [214] S. Watanabe, T. Hori, S. Kim, J. R. Hershey, and T. Hayashi, “Hybrid CTC Attention Architecture for End-to-End Speech Recognition,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 11, no. 8, pp. 1240–1253, 2017.
  * [215] H. Seki, T. Hori, S. Watanabe, N. Moritz, and J. Le Roux, “Vectorized Beam Search for CTC-Attention-Based Speech Recognition,” in _Proc. Interspeech_ , Brighton, UK, May 2019, pp. 3825–3829.
  * [216] A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates, and A. Y. Ng, “Deep Speech: Scaling up End-to-End Speech Recognition,” Dec. 2014, arXiv:1412.5567. [Online]. Available: https://arxiv.org/abs/1412.5567
  * [217] T. Hori, S. Watanabe, and J. R. Hershey, “Multi-Level Language Modeling and Decoding for Open Vocabulary End-to-End Speech Recognition,” in _Proc. IEEE ASRU_. Okinawa, Japan: IEEE, Dec. 2017, pp. 287–293.
  * [218] Y. Wang, T. Chen, H. Xu, S. Ding, H. Lv, Y. Shao, N. Peng, L. Xie, S. Watanabe, and S. Khudanpur, “Espresso: A Fast End-to-End Neural Speech Recognition Toolkit,” in _Proc. IEEE ASRU_ , Sentosa, Singapore, Dec. 2019, pp. 136–143.
  * [219] Z. Tüske, K. Audhkhasi, and G. Saon, “Advancing Sequence-to-Sequence Based Speech Recognition,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 3780–3784.
  * [220] J. Drexler and J. Glass, “Subword Regularization and Beam Search Decoding for End-to-End Automatic Speech Recognition,” in _Proc. IEEE ICASSP_. Brighton, UK: IEEE, May 2019, pp. 6266–6270.
  * [221] T. N. Sainath, R. Pang, D. Rybach, Y. He, R. Prabhavalkar, W. Li, M. Visontai, Q. Liang, T. Strohman, Y. Wu, I. McGraw, and C.-C. Chiu, “Two-Pass End-to-End Speech Recognition,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 2773–2777.
  * [222] Z. Yao, D. Wu, X. Wang, B. Zhang, F. Yu, C. Yang, Z. Peng, X. Chen, L. Xie, and X. Lei, “WeNet: Production Oriented Streaming and Non-Streaming End-to-End Speech Recognition Toolkit,” Brno, Czechia, pp. 4054–4058, Sep. 2021.
  * [223] D. Wu, B. Zhang, C. Yang, Z. Peng, W. Xia, X. Chen, and X. Lei, “U2++: Unified Two-Pass Bidirectional End-to-End Model for Speech Recognition,” Dec. 2021, arXiv:2106.05642. [Online]. Available: https://arxiv.org/abs/2106.05642
  * [224] M. Zapotoczny, P. Pietrzak, A. Lancucki, and J. Chorowski, “Lattice Generation in Attention-Based Speech Recognition Models,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 2225–2229.
  * [225] J. Kim, Y. Lee, and E. Kim, “Accelerating RNN Transducer Inference via Adaptive Expansion Search,” _IEEE Signal Processing Letters_ , vol. 27, pp. 2019–2023, 2020.
  * [226] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: A system for Large-Scale Machine Learning,” in _Proc. OSDI_ , Savannah, GA, Nov. 2016, pp. 265–283.
  * [227] M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “FAIRSEQ: A Fast, Extensible Toolkit for Sequence Modeling,” in _Proc. NAACL_ , Minneapolis, MN, Jun. 2019, pp. 48–53.
  * [228] P. Doetsch, A. Zeyer, P. Voigtlaender, I. Kulikov, R. Schlüter, and H. Ney, “RETURNN: The RWTH Extensible Training Framework for Universal Recurrent Neural Networks,” in _Proc. IEEE ICASSP_. New Orleans, LA: IEEE, Mar. 2017, pp. 5345–5349.
  * [229] T. Mikolov, M. Karafiát, L. Burget, J. Černockỳ, and S. Khudanpur, “Recurrent Neural Network Based Language Model,” in _Proc. Interspeech_ , Makuhari, Japan, Sep. 2010, pp. 1045–1048.
  * [230] S. F. Chen and J. Goodman, “An Empirical Study of Smoothing Techniques for Language Modeling,” in _Proc. ACL_ , Santa Cruz, CA, Jun. 1996, pp. 310–318.
  * [231] M. Sundermeyer, R. Schlüter, and H. Ney, “LSTM Neural Networks for Language Modeling,” in _Proc. Interspeech_ , Portland, OR, Sep. 2012, pp. 194–197.
  * [232] Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, “Language Modeling with Gated Convolutional Networks,” in _Proceedings of the 34th International Conference on Machine Learning-Volume 70_. JMLR. org, 2017, pp. 933–941.
  * [233] N. Zeghidour, Q. Xu, V. Liptchinsky, N. Usunier, G. Synnaeve, and R. Collobert, “Fully Convolutional Speech Recognition,” Feb. 2018, arXiv:1812.06864. [Online]. Available: https://arxiv.org/abs/1812.06864
  * [234] R. Al-Rfou, D. Choe, N. Constant, M. Guo, and L. Jones, “Character-Level Language Modeling with Deeper Self-Attention,” in _Proc. AIII_ , vol. 33, Honolulu, Hawaii, Feb. 2019, pp. 3159–3166.
  * [235] K. Irie, A. Zeyer, R. Schlüter, and H. Ney, “Language Modeling with Deep Transformers,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 3905–3909.
  * [236] Z. Dai, Z. Yang, Y. Yang, J. G. Carbonell, Q. Le, and R. Salakhutdinov, “Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context,” in _Proc. ACL_ , Florence, Italy, Jul. 2019, pp. 2978–2988.
  * [237] J. W. Rae, A. Potapenko, S. M. Jayakumar, and T. P. Lillicrap, “Compressive Transformers for Long-Range Sequence Modelling,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 6154–6158, 2020.
  * [238] C. Gulcehre, O. Firat, K. Xu, K. Cho, L. Barrault, H.-C. Lin, F. Bougares, H. Schwenk, and Y. Bengio, “On Using Monolingual Corpora in Neural Machine Translation,” Jun. 2015, arXiv:1503.03535. [Online]. Available: https://arxiv.org/abs/1503.03535
  * [239] A. Sriram, H. Jun, S. Satheesh, and A. Coates, “Cold Fusion: Training Seq2Seq Models Together with Language Models,” in _Proc. Interspeech_ , Hyderabad, India, Sep. 2018, pp. 387–391.
  * [240] C. Shan, C. Weng, G. Wang, D. Su, M. Luo, D. Yu, and L. Xie, “Component Fusion: Learning Replaceable Language Model Component for End-to-End Speech Recognition System,” in _Proc. IEEE ICASSP_. Brighton, UK: IEEE, May 2019, pp. 5361–5635.
  * [241] E. McDermott, H. Sak, and E. Variani, “A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognition,” in _Proc. IEEE ASRU_ , Sentosa, Singapore, Dec. 2019, pp. 434–441.
  * [242] Z. Meng, S. Parthasarathy, E. Sun, Y. Gaur, N. Kanda, L. Lu, X. Chen, R. Zhao, J. Li, and Y. Gong, “Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognition,” in _Proc. IEEE SLT_ , Shenzhen , China, Dec. 2020, pp. 243–250.
  * [243] W. Zhou, Z. Zheng, R. Schlüter, and H. Ney, “On Language Model Integration for RNN Transducer based Speech Recognition,” in _Proc. IEEE ICASSP_ , Singapore, May 2022, pp. 8407–8411, arXiv:2110.06841. [Online]. Available: http://arxiv.org/abs/2110.06841
  * [244] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” in _Proc. ACL_ , Florence, Italy, Jul. 2019, pp. 4171–4186.
  * [245] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language Models are Unsupervised Multitask Learners,” 2019, openAI blog. [Online]. Available: https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
  * [246] J. Salazar, D. Liang, T. Q. Nguyen, and K. Kirchhoff, “Masked Language Model Scoring,” in _Proc. ACL_ , Jul. 2020, pp. 2699–2712.
  * [247] S. Kim, S. Dalmia, and F. Metze, “Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion,” in _Proc. ACL_ , Florence, Italy, Jul. 2019, pp. 1131–1141.
  * [248] A. Zeyer, E. Beck, R. Schlüter, and H. Ney, “CTC in the Context of Generalized Full-Sum HMM Training,” in _Proc. Interspeech_ , Stockholm, Sweden, Aug. 2017, pp. 944–948.
  * [249] T. Raissi, W. Zhou, S. Berger, R. Schlüter, and H. Ney, “HMM vs. CTC for Automatic Speech Recognition: Comparison Based on Full-Sum Training from Scratch,” in _Proc. IEEE SLT_ , Doha, Qatar, Jan. 2023, arXiv:2210.09951. [Online]. Available: http://arxiv.org/abs/2210.09951
  * [250] A. Hannun, A. Lee, Q. Xu, and R. Collobert, “Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 3785–3789.
  * [251] M. Li, M. Liu, and H. Masanori, “End-to-End Speech Recognition with Adaptive Computation Steps,” in _Proc. IEEE ICASSP_ , Brighton, UK, May 2019, pp. 6246–6250.
  * [252] P. Bahar, N. Makarov, A. Zeyer, R. Schüter, and H. Ney, “Exploring a Zero-Order Direct HMM Based on Latent Attention for Automatic Speech Recognition,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 7854–7858.
  * [253] Z. Huang, G. Zweig, and B. Dumoulin, “Cache Based Recurrent Neural Network Language Model Inference for First Pass Speech Recognition,” in _Proc. IEEE ICASSP_ , Florence, Italy, May 2014, pp. 6354–6358.
  * [254] J. Jorge, A. Giménez, J. Iranzo-Sánchez, J. Civera, A. Sanchis, and A. Juan, “Real-Time One-Pass Decoder for Speech Recognition Using LSTM Language Models,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 3820–3824.
  * [255] W. Zhou, R. Schlüter, and H. Ney, “Full-Sum Decoding for Hybrid HMM Based Speech Recognition Using LSTM Language Model,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 7834–7838.
  * [256] P. Sountsov and S. Sarawagi, “Length Bias in Encoder Decoder Models and a Case for Global Conditioning,” in _Proc. EMNLP_ , Austin, TX, Nov. 2016, pp. 1516–1525.
  * [257] K. Murray and D. Chiang, “Correcting Length Bias in Neural Machine Translation,” in _Proc. WMT_ , Brussels, Belgium, Oct. 2018, pp. 212–223.
  * [258] F. Stahlberg and B. Byrne, “On NMT Search Errors and Model Errors: Cat Got Your Tongue?” in _Proc. EMNLP_. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 3354–3360.
  * [259] L. Lu, X. Zhang, and S. Renals, “On Training the Recurrent Neural Network Encoder-Decoder for Large Vocabulary End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_ , Shanghai, China, Mar. 2016, pp. 5060–5064.
  * [260] Y. Zhang, W. Chan, and N. Jaitly, “Very Deep Convolutional Networks for End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_ , New Orleans, LA, Mar. 2017, pp. 4845–4849.
  * [261] S. Toshniwal, H. Tang, L. Lu, and K. Livescu, “Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder based Speech Recognition,” Apr. 2017, arXiv:1704.01631. [Online]. Available: https://arxiv.org/abs/1704.01631
  * [262] A. Renduchintala, S. Ding, M. Wiesner, and S. Watanabe, “Multi-Modal Data Augmentation for End-to-End ASR,” in _Proc. Interspeech_ , Hyderabad, India, Mar. 2018, pp. 2394–2398.
  * [263] C. Weng, J. Cui, G. Wang, J. Wang, C. Yu, D. Su, and D. Yu, “Improving Attention Based Sequence-to-Sequence Models for End-to-End English Conversational Speech Recognition,” in _Proc. Interspeech_ , Hyderabad, India, Sep. 2018, pp. 761–765.
  * [264] D. Le, X. Zhang, W. Zheng, C. Fügen, G. Zweig, and M. L. Seltzer, “From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition,” in _Proc. IEEE ASRU_ , Sentosa, Singapore, Dec. 2019, pp. 457–464.
  * [265] S. Kanthak and H. Ney, “Context-Dependent Acoustic Modeling Using Graphemes for Large Vocabulary Speech Recognition,” in _Proc. IEEE ICASSP_ , Orlando, FL, May 2002, pp. 845–848.
  * [266] N. Deshmukh, A. Ganapathiraju, and J. Picone, “Hierarchical Search for Large-Vocabulary Conversational Speech Recognition: Working Toward a Solution to the Decoding Problem,” _IEEE Signal Processing Magazine_ , vol. 16, no. 5, pp. 84–107, 1999.
  * [267] L. Nguyen and R. Schwartz, “Single-Tree Method for Grammar-Directed Search,” in _Proc. IEEE ICASSP_ , vol. 2, Phoenix, AZ, Mar. 1999, pp. 613–616.
  * [268] L. Sarı, N. Moritz, T. Hori, and J. Le Roux, “Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 2–6.
  * [269] F. Weninger, J. Andrés-Ferrer, X. Li, and P. Zhan, “Listen, Attend, Spell and Adapt: Speaker Adapted Sequence-to-Sequence ASR,” in _Proc. Interspeech_. Graz, Austria: ISCA, Sep. 2019, pp. 3805–3809.
  * [270] Z. Meng, Y. Gaur, J. Li, and Y. Gong, “Speaker Adaptation for Attention-Based End-to-End Speech Recognition,” in _Proc. Interspeech_. Graz, Austria: ISCA, Sep. 2019, pp. 241–245.
  * [271] N. Tomashenko and Y. Estève, “Evaluation of Feature-Space Speaker Adaptation for End-to-End Acoustic Models,” in _Proc. LREC_. Miyazaki, Japan: ELRA, May 2018, pp. 3163–3170.
  * [272] A. Zeyer, A. Merboldt, W. Michel, R. Schlüter, and H. Ney, “Librispeech Transducer Model with Internal Language Model Prior Correction,” in _Proc. Interspeech_ , Brno, Czech Republic, Apr. 2021, pp. 2052–2056.
  * [273] L. R. Bahl, F. Jelinek, and R. L. Mercer, “A Maximum Likelihood Approach to Continuous Speech Recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 5, no. 2, pp. 179–190, Mar. 1983.
  * [274] J. Makhoul and R. Schwartz, “State of the Art in Continuous Speech Recognition,” _Proc. NAS_ , vol. 92, no. 22, pp. 9956–9963, Oct. 1995\.
  * [275] D. Klakow and J. Peters, “Testing the Correlation of Word Error Rate and Perplexity,” _Speech Communication_ , vol. 38, no. 1, pp. 19–28, 2002.
  * [276] M. Sundermeyer, H. Ney, and R. Schlüter, “From Feedforward to Recurrent LSTM Neural Networks for Language Modeling,” _IEEE/ACM Trans. Audio, Speech, and Language Processing_ , vol. 23, no. 3, pp. 517–529, Mar. 2015.
  * [277] Y. Tachioka and S. Watanabe, “Discriminative Method for Recurrent Neural Network Language Models,” in _Proc. IEEE ICASSP_ , South Brisbane, Australia, Apr. 2015, pp. 5386–5390.
  * [278] T. Hori, C. Hori, S. Watanabe, and J. R. Hershey, “Minimum Word Error Training of Long Short-Term Memory Recurrent Neural Network Language Models for Speech Recognition,” in _Proc. IEEE ICASSP_ , Shanghai, China, Mar. 2016, pp. 5990–5994.
  * [279] T. N. Sainath, R. Prabhavalkar, S. Kumar, S. Lee, A. Kannan, D. Rybach, V. Schogol, P. Nguyen, B. Li, Y. Wu, Z. Chen, and C.-C. Chiu, “No Need for a Lexicon? Evaluating the Value of the Pronunciation Lexica in End-to-End Models,” in _Proc. IEEE ICASSP_ , Calgary, Alberta, Canada, Apr. 2018, pp. 5859–5863.
  * [280] C. Wooters and A. Stolcke, “Multiple-Pronunciation Lexical Modeling in a Speaker Independent Speech Understanding System,” in _Proc. ICSLP_ , Yokohama, Japan, Sep. 1994, pp. 1363–1366.
  * [281] I. McGraw, I. Badr, and J. R. Glass, “Learning Lexicons From Speech Using a Pronunciation Mixture Model,” _IEEE/ACM Trans. Audio, Speech, and Language Processing_ , vol. 21, no. 2, pp. 357–366, 2012.
  * [282] A. Senior, G. Heigold, M. Bacchiani, and H. Liao, “GMM-Free DNN Acoustic Model Training,” in _Proc. IEEE ICASSP_ , Florence, Italy, May 2014, pp. 5602–5606. [Online]. Available: https://doi.org/10.1109/ICASSP.2014.6854675
  * [283] G. Gosztolya, T. Grósz, and L. Tóth, “GMM-Free Flat Start Sequence-Discriminative DNN Training,” in _Proc. Interspeech_ , N. Morgan, Ed. San Francisco, CA: ISCA, Sep. 2016, pp. 3409–3413. [Online]. Available: https://doi.org/10.21437/Interspeech.2016-391
  * [284] H. Hadian, H. Sameti, D. Povey, and S. Khudanpur, “Flat-Start Single-Stage Discriminatively Trained HMM-Based Models for ASR,” _IEEE/ACM Trans. Audio, Speech, and Language Processing_ , vol. 26, no. 11, pp. 1949–1961, 2018\.
  * [285] H. Soltau, B. Kingsbury, L. Mangu, D. Povey, G. Saon, and G. Zweig, “The IBM 2004 Conversational Telephony System for Rich Transcription,” in _Proc. IEEE ICASSP_ , Philadelphia, PA, Mar. 2005, pp. 205–208.
  * [286] H. Hadian, D. Povey, H. Sameti, J. Trmal, and S. Khudanpur, “Improving LF-MMI Using Unconstrained Supervisions for ASR,” in _Proc. IEEE SLT_ , Athens, Greece, Dec. 2018, pp. 43–47. [Online]. Available: https://doi.org/10.1109/SLT.2018.8639684
  * [287] N. Kanda, Y. Fujita, and K. Nagamatsu, “Lattice-Free State-Level Minimum Bayes Risk Training of Acoustic Models,” in _Proc. Interspeech_ , B. Yegnanarayana, Ed. Hyderabad, India: ISCA, Sep. 2018, pp. 2923–2927. [Online]. Available: https://doi.org/10.21437/Interspeech.2018-79
  * [288] S. J. Young and P. C. Woodland, “The Use of State Tying in Continuous Speech Recognition,” in _Proc. Eurospeech_ , Berlin, Germany, Dec. 1993, pp. 2203–2206.
  * [289] S. Wiesler, G. Heigold, M. Nußbaum-Thom, R. Schlüter, and H. Ney, “A Discriminative Splitting Criterion for Phonetic Decision Trees,” in _Proc. Interspeech_ , Makuhari, Japan, Sep. 2010, pp. 54–57, one of shortlist for Best Student Paper Award.
  * [290] H. Xu, S. Ding, and S. Watanabe, “Improving End-to-End Speech Recognition with Pronunciation-Assisted Sub-Word Modeling,” in _Proc. IEEE ICASSP_ , Brighton, UK, Sep. 2019, pp. 7110–7114.
  * [291] W. Zhou, M. Zeineldeen, Z. Zheng, R. Schlüter, and H. Ney, “Acoustic Data-Driven Subword Modeling for End-to-End Speech Recognition,” in _Proc. Interspeech_ , Brno, Czechia, Aug. 2021, pp. 2886–2890.
  * [292] T. Raissi, E. Beck, R. Schlüter, and H. Ney, “Towards Consistent Hybrid HMM Acoustic Modeling,” Apr. 2021, arXiv:2104.02387. [Online]. Available: https://arxiv.org/abs/2104.02387
  * [293] M. Zeineldeen, A. Zeyer, W. Zhou, T. Ng, R. Schlüter, and H. Ney, “A Systematic Comparison of Grapheme-Based vs. Phoneme-Based Label Units for Encoder-Decoder-Attention Models,” Nov. 2020, arXiv:2005.09336. [Online]. Available: https://arxiv.org/abs/2005.09336
  * [294] C. Lüscher, E. Beck, K. Irie, M. Kitza, W. Michel, A. Zeyer, R. Schlüter, and H. Ney, “RWTH ASR Systems for LibriSpeech: Hybrid vs Attention,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 231–235.
  * [295] D. Park, Y. Zhang, Y. Jia, W. Han, C.-C. Chiu, B. Li, Y. Wu, and Q. Le, “Improved Noisy Student Training for Automatic Speech Recognition,” in _Proc. Interspeech_ , Shanghai, China, Oct. 2020, pp. 2817–2821.
  * [296] D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 2613–2617.
  * [297] W. Zhou, W. Michel, K. Irie, M. Kitza, R. Schlüter, and H. Ney, “The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugment,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 7839–7843.
  * [298] J. Cui, B. Kingsbury, B. Ramabhadran, A. Sethy, K. Audhkhasi, X. Cui, E. Kislal, L. Mangu, M. Nussbaum-Thom, M. Picheny, Z. Tüske, P. Golik, R. Schlüter, H. Ney, M. J. F. Gales, K. M. Knill, A. Ragni, H. Wang, and P. Woodland, “Multilingual Representations for Low Resource Speech Recognition and Keyword Search,” in _Proc. IEEE ASRU_ , Scottsdale, AZ, Dec. 2015, pp. 259–266.
  * [299] O. Adams, M. Wiesner, S. Watanabe, and D. Yarowsky, “Massively Multilingual Adversarial Speech Recognition,” in _Proc. NAACL_ , Minneapolis, MN, Jun. 2019, pp. 96–108.
  * [300] A. Kannan, A. Datta, T. N. Sainath, E. Weinstein, B. Ramabhadran, Y. Wu, A. Bapna, Z. Chen, and S. Lee, “Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model,” in _Proc. Interspeech_ , Graz, Austria, Sep. 2019, pp. 2130–2134.
  * [301] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an ASR Corpus Based on Public Domain Audio Books,” in _Proc. IEEE ICASSP_ , Queensland, Australia, Apr. 2015, pp. 5206–5210.
  * [302] J. Godfrey, E. Holliman, and J. McDaniel, “SWITCHBOARD: Telephone Speech Corpus for Research and Development,” in _Proc. IEEE ICASSP_ , vol. 1, San Francisco, CA, Mar. 1992, pp. 517–520 vol.1.
  * [303] Y. Wang, A. Mohamed, D. Le, C. Liu, A. Xiao, J. Mahadeokar, H. Huang, A. Tjandra, X. Zhang, F. Zhang, C. Fuegen, G. Zweig, and M. L. Seltzer, “Transformer-Based Acoustic Modeling for Hybrid Speech Recognition,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 6874–6878.
  * [304] G. Synnaeve, Q. Xu, J. Kahn, E. Grave, T. Likhomanenko, V. Pratap, A. Sriram, V. Liptchinsky, and R. Collobert, “End-to-End ASR: from Supervised to Semi-Supervised Learning with Modern Architectures,” in _Proc. ICML_ , Jul. 2020, arXiv:1911.08460.
  * [305] K. Kim, F. Wu, Y. Peng, J. Pan, P. Sridhar, K. J. Han, and S. Watanabe, “E-branchformer: Branchformer with enhanced merging for speech recognition,” in _Proc. IEEE SLT_ , Doha, Qatar, Jan. 2023, to appear; arXiv:2210.00077. [Online]. Available: https://arxiv.org/abs/2210.00077
  * [306] M. Kitza, P. Golik, R. Schlüter, and H. Ney, “Cumulative Adaptation for BLSTM Acoustic Models,” in _Interspeech_ , Graz, Austria, Sep. 2019, pp. 754–758.
  * [307] A. Zeyer, P. Bahar, K. Irie, R. Schlüter, and H. Ney, “A Comparison of Transformer and LSTM Encoder Decoder Models for ASR,” in _Proc. IEEE ASRU_ , Sentosa, Singapore, Dec. 2019, pp. 8–15.
  * [308] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,” _IEEE/ACM Trans. Audio, Speech, and Language Processing_ , vol. 19, pp. 3451–3460, 2021.
  * [309] E. G. Ng, C.-C. Chiu, Y. Zhang, and W. Chan, “Pushing the Limits of Non-Autoregressive Speech Recognition,” in _Proc. Interspeech_ , Brno, Czechia, Sep. 2021, pp. 3725–2729.
  * [310] J. Kahn, M. Riviere, W. Zheng, E. Kharitonov, Q. Xu, P. E. Mazare, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux, “Libri-Light: A Benchmark for ASR with Limited or no Supervision,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, May 2020, pp. 7669–7673.
  * [311] C.-C. Chiu, T. N. Sainath, Y. Wu, R. Prabhavalkar, P. Nguyen, Z. Chen, A. Kannan, R. J. Weiss, K. Rao, E. Gonina, N. Jaitly, B. Li, J. Chorowski, and M. Bacchiani, “State-of-the-Art Speech Recognition with Sequence-to-Sequence Models,” in _Proc. IEEE ICASSP_ , Calgary, Alberta, Canada, Apr. 2018, pp. 4774–4778.
  * [312] K. Kim, K. Lee, D. Gowda, J. Park, S. Kim, S. Jin, Y.-Y. Lee, J. Yeo, D. Kim, S. Jung, J. Lee, M. Han, and C. Kim, “Attention Based On-Device Streaming Speech Recognition with Large Speech Corpus,” in _Proc. IEEE ASRU_ , Sentosa, Singapore, Dec. 2019, pp. 956–963.
  * [313] R. Hsiao, D. Can, T. Ng, R. Travadi, and A. Ghoshal, “Online Automatic Speech Recognition with Listen, Attend and Spell Model,” _IEEE Signal Processing Letters_ , vol. 27, pp. 1889–1893, 2020.
  * [314] Y. Shi, Y. Wang, C. Wu, C.-F. Yeh, J. Chan, F. Zhang, D. Le, and M. Seltzer, “Emformer: Efficient Memory Transformer based Acoustic Model for Low Latency Streaming Speech Recognition,” in _Proc. IEEE ICASSP_. Toronto, Ontario, Canada: IEEE, Jun. 2021, pp. 6783–6787.
  * [315] X. Chen, Y. Wu, Z. Wang, S. Liu, and J. Li, “Developing Real-Time Streaming Transformer Transducer for Speech Recognition on Large-Scale Dataset,” in _Proc. IEEE ICASSP_. Toronto, Ontario, Canada: IEEE, Jun. 2021, pp. 5904–5908.
  * [316] T. N. Sainath, Y. He, B. Li, A. Narayanan, R. Pang, A. Bruguier, S.-y. Chang, W. Li, R. Alvarez, Z. Chen, C.-C. Chiu, D. Garcia, A. Gruenstein, K. Hu, M. Jin, A. Kannan, Q. Liang, I. McGraw, C. Peyser, R. Prabhavalkar, G. Pundak, D. Rybach, Y. Shangguan, Y. Sheth, T. Strohman, M. Visontai, Y. Wu, Y. Zhang, and D. Zhao, “A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency,” in _Proc. IEEE ICASSP_ , Barcelona, Spain, may 2020, pp. 6059–6063.
  * [317] B. Li, A. Gulati, J. Yu, T. N. Sainath, C.-C. Chiu, A. Narayanan, S.-Y. Chang, R. Pang, Y. He, J. Qin, W. Han, Q. Liang, Y. Zhang, T. Strohman, and Y. Wu, “A Better and Faster End-to-End Model for Streaming ASR,” in _Proc. IEEE ICASSP_ , Toronto, Ontario, Canada, Jun. 2021, pp. 5634–5638.
  * [318] T. N. Sainath, Y. He, A. Narayanan, R. Botros, R. Pang, D. Rybach, C. Allauzen, E. Variani, J. Qin, Q.-N. Le-The, S.-Y. Chang, B. Li, A. Gulati, J. Yu, C.-C. Chiu, D. Caseiro, W. Li, Q. Liang, and P. Rondon, “An Efficient Streaming Non-Recurrent On-Device End-to-End Model with Improvements to Rare-Word Modeling,” in _Proc. Interspeech_ , Brno, Czechia, Sep. 2021, pp. 1777–1781.
  * [319] N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers _et al._ , “In-Datacenter Performance Analysis of a Tensor Processing Unit,” in _Proc. of the 44 th Annual International Symposium on Computer Architecture_, Toronto, Ontario, Canada, Jun. 2017, pp. 1–12.
  * [320] A. Bapna, Y.-A. Chung, N. Wu, , A. Gulati, Y. Jia, J. H. Clark, M. Johnson, J. Riesa, A. Conneau, and Y. Zhang, “SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training,” Oct. 2021, arXiv:2110.10329. [Online]. Available: https://arxiv.org/abs/2110.10329
  * [321] A. Bapna, C. Cherry, Y. Zhang, Y. Jia, M. Johnson, Y. Cheng, S. Khanuja, J. Riesa, and A. Conneau, “mSLAM: Massively Multilingual Joint Pre-Training for Speech and Text,” Feb. 2022, arXiv:2202.01374. [Online]. Available: https://arxiv.org/abs/2202.01374
  * [322] Y. Tang, H. Gong, N. Dong, C. Wag, W. Hsu, J. Gu, A. Baevski, X. Li, A. Mohamed, M. Auli, and J. Pino, “Unified Speech-Text Pre-training for Speech Translation and Recognition,” in _Proc. ACL_ , Dublin, Ireland, May 2022, pp. 1488–1499, arXiv:2204.05409. [Online]. Available: https://arxiv.org/abs/2204.05409
  * [323] Y.-A. Chung, C. Zhu, and M. Zeng, “SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding,” in _Proc. NAACL_ , Jun. 2021, pp. 1897–1907, arXiv:2010.02295. [Online]. Available: https://arxiv.org/abs/2010.02295
  * [324] J. Ao, R. Wang, L. Zhou, C. Wang, S. Ren, Y. Wu, S. Liu, T. Ko, Q. Li, Y. Zhang, Z. Wei, Y. Qian, J. Li, and F. Wei, “SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing,” in _Proc. ACL_ , Dublin, Ireland, May 2022, pp. 5723–5738, arXiv:2110.07205. [Online]. Available: https://arxiv.org/abs/2110.07205
  * [325] S. Thomas, H. J. Kuo, B. Kingsbury, and G. Saon, “Towards Reducing the Need for Speech Training Data to Build Spoken Language Understanding Systems,” in _Proc. IEEE ICASSP_ , Singapore, May 2022, pp. 7932–7936, arXiv:2203.00006. [Online]. Available: https://arxiv.org/abs/2203.00006
  * [326] T. N. Sainath, R. Prabhavalkar, A. Bapna, Y. Zhang, Z. Huo, Z. Chen, B. Li, W. Wang, and T. Strohman, “JOIST: A joint speech and text streaming model for ASR,” in _Proc. IEEE SLT_ , Doha, Qatar, Jan. 2023, to appear; arXiv:2210.07353. [Online]. Available: https://arxiv.org/abs/2210.07353
  * [327] T. Hori, R. Astudillo, T. Hayashi, Y. Zhang, S. Watanabe, and J. Le Roux, “Cycle-Consistency Training for End-to-End Speech Recognition,” in _Proc. IEEE ICASSP_ , Brighton, UK, May 2019, pp. 6271–6275.
  * [328] T. Ochiai, S. Watanabe, T. Hori, and J. R. Hershey, “Multichannel End-to-End Speech Recognition,” in _Proc. ICML_. Sydney, Australia: PMLR, Aug. 2017, pp. 2632–2641.
  * [329] X. Chang, W. Zhang, Y. Qian, J. Le Roux, and S. Watanabe, “MIMO-Speech: End-to-End Multi-Channel Multi-Speaker Speech Recognition,” in _Proc. IEEE ASRU_. Sentosa, Singapore: IEEE, Dec. 2019, pp. 237–244.
  * [330] J. Li, “Recent Advances in End-to-End Automatic Speech Recognition,” Nov. 2021, preprint arXiv:2111.01690. [Online]. Available: https://arxiv.org/abs/2111.01690

|  Rohit Prabhavalkar Rohit Prabhavalkar received his PhD in Computer Science
and Engineering from The Ohio State University, USA, in 2013. Following his
PhD, Rohit joined the Speech Technologies group at Google where he is
currently a Staff Research Scientist. At Google, his research has focused
primarily on developing compact acoustic models which can run efficiently on
mobile devices, and on developing improved end-to-end automatic speech
recognition systems. Rohit has co-authored over 50 refereed papers, which have
received two best paper awards (ASRU 2017; ICASSP 2018). He currently serves
as a member of the IEEE Speech and Language Processing Technical Committee
(2018–2024) and as an associate editor of the IEEE/ACM Transactions on Audio,
Speech, and Language Processing.
---|---
|  Takaaki Hori received his PhD degree in system and information engineering
from Yamagata University, Yonezawa, Japan, in 1999. From 1999 to 2015, he had
been engaged in researches on speech recognition and spoken language
processing at Cyber Space Laboratories and Communication Science Laboratories
in Nippon Telegraph and Telephone (NTT) Corporation, Japan. From 2015 to 2021,
he was a Senior Principal Research Scientist at Mitsubishi Electric Research
Laboratories (MERL), USA. He is currently a Machine Learning Researcher at
Apple. His research interests include automatic speech recognition, spoken
language understanding, and language modeling. He served as a member of the
IEEE Speech and Language Processing Technical Committee (2020–2022).
---|---
|  Tara Sainath received her PhD in Electrical Engineering and Computer
Science from MIT in 2009. The main focus of her PhD work was in acoustic
modeling for noise robust speech recognition. After her PhD, she spent 5 years
at the Speech and Language Algorithms group at IBM T.J. Watson Research
Center, before joining Google Research. She has served as a Program Chair for
ICLR in 2017 and 2018. Also, she has co-organized numerous special sessions
and workshops, including Interspeech 2010, ICML 2013, Interspeech 2016 and
ICML 2017. In addition, she is a member of the IEEE Speech and Language
Processing Technical Committee (SLTC) as well as the Associate Editor for
IEEE/ACM Transactions on Audio, Speech, and Language Processing.
---|---
|  Ralf Schlüter Ralf Schlüter received his Dr.rer.nat. degree in Computer
Science in 2000 and habilitated in Computer Science in 2019, both at RWTH
Aachen University. In May 1996, Ralf Schlüter joined the Computer Science
Department at RWTH Aachen University, where he currently is Lecturer and
Academic Director, leading the Automatic Speech Recognition Group at the Chair
Computer Science 6 – Machine Learning and Human Language Technology. In 2019,
Ralf also joined AppTek GmbH Aachen as Senior Researcher. His research
interests cover sequence classification, specifically all aspects of automatic
speech recognition, decision theory, stochastic modeling, and signal analysis.
Ralf served as Subject Editor for Speech Communication (2013-2019).
---|---
|  Shinji Watanabe Shinji Watanabe is an Associate Research Professor at Johns
Hopkins University, Baltimore, MD, USA. He received his PhD degree in 2006
from Waseda University, Tokyo, Japan. He was a research scientist at NTT
Communication Science Laboratories, Kyoto, and a Senior Principal Research
Scientist at Mitsubishi Electric Research Laboratories (MERL). His research
interests include automatic speech recognition, speech enhancement, and
machine learning for speech and language processing. He served an Associate
Editor of the IEEE Transactions on Audio Speech and Language Processing, and
is a member of several technical committees including the IEEE Signal
Processing Society Speech and Language Technical Committee.
---|---


# Anomalous Quantum Hall Bilayer Effect

Gurjyot Sethi  Feng Liu  Department of Materials Science and Engineering,
University of Utah, Salt Lake City, Utah 84112, USA

###### Abstract

Quantum Hall bilayers (QHB) are two-component fractional quantum Hall (FQH)
systems that provide a unique platform to realize exotic Abelian/non-Abelian
FQH and excitonic condensate states, but in the presence of a high magnetic
field. Here, we show that these states can be realized in an analogue of
bilayer Kagome lattice with flat bands (FBs) in each layer, leading to
anomalous quantum Hall bilayer effect, without magnetic field. Using exact
diagonalization of finite size bilayer Kagome lattices, we demonstrate the
stabilization of excitonic condensate Halperin’s $(1,1,1)$ and (3,3,3) state
at the total filling $v_{T}=1$ and $v_{T}=1/3$ of the two FBs, respectively.
We further show that by tuning the interlayer tunneling at $v_{T}=2/3$, one
can expect a phase transition from Halperin’s $(3,3,1)$ state to particle-hole
conjugate of Laughlin’s $1/3$ state, as previously observed in QHB systems.
Our work significantly enriches the field of FB physics by demonstrating
bilayer FB systems as an attractive avenue for realizing exotic QHB states
including non-Abelian anyons without magnetic field.

††preprint: APS/123-QED

When two-dimensional (2D) free electron gas is subject to a magnetic field,
electron-electron interactions in the Landau levels of electrons lead to
remarkable fractional quantum Hall (FQH) effect with fractionally quantized
Hall conductance [1, 2]. A peculiar property of emergent excitations of FQH
state is that they carry fractional charges [3] following anyonic statistics
in 2D [4, 5]. Possibility of non-Abelian fractional statistics [6] obeyed by
FQH states are particularly interesting owing to their potential in
topological quantum computation [7] and are expected to arise in a single-
component FQH system due to electron-electron interactions in the excited
Landau levels of free electron gas [8, 9, 10].

Beyond single-component FQH, multi-component FQH systems offer a richer and
more exotic quantum phase diagram with extra degrees of freedom allowing for
additional tuning parameters [11, 12, 13, 14]. Quantum Hall Bilayers (QHB),
such as GaAs quantum wells (QWs) [15, 16, 17, 18], are spectacular examples of
multi-component FQH systems. In such systems either a wide QW or a double QW
can be mapped to two layers of free electron gas separated by a finite
distance forming the two components that lead to exotic multi-component
quantum Hall plateaus [15, 16, 17, 18]. One of the most notable plateaus
manifests at $v_{T}=1$ (Halperin’s $(1,1,1)$ state) where interlayer excitonic
coherence is observed by counter-flow measurements [19, 20, 21]. In addition,
QHBs have generated a lot of theoretical [22, 23, 24, 25, 26, 27, 28]
interest, and are known to host a plethora of Abelian and non-Abelian FQH
phase transitions owing to exceptional tunability of parameters; for example,
tuning the interlayer tunneling can induce transitions from Abelian Halperin’s
$(3,3,0)$ to particle-hole conjugate of $1/3$ Laughlin’s state at $v_{T}=2/3$
[28], as well as from Abelian Halperin’s $(3,3,1)$ to non-Abelian Moore-Read
Pfaffian state at $v_{T}=1/2$ [25].

Interestingly, in the past decade 2D lattices with topologically non-trivial
Chern FB [29, 30] have been shown to stabilize fractional quantum Hall effect
(FQH) without magnetic field, giving rise to the so-called fractional Chern
insulator (FCI) [31, 32, 33, 34]. In this sense, non-trivial FBs can be
considered as solid-state analogue of dispersion-less Landau levels of the
free electron gas. At fractional filling of FBs, ground state of the system
mimics the Laughlin’s FQH state with features like ground-state degeneracy on
a torus [31, 32, 33, 34]] spectral flow under flux insertion [35, 36],
quasihole statistics [33], and entanglement spectra [37]. On the other hand,
however, the realization of non-Abelian anyons in FCIs is not straightforward
since in a single-component FQH they arise at partial filling of excited
Landau levels ($v_{T}>1$) [8, 9, 10] while topological FB mimics only the
lowest Landau level of free electron gas. A largely unexplored research
direction, which is actually a natural extension to FCI, is the realization of
exotic multi-component quantum Hall bilayer states without magnetic field in
topological FBs. This is especially appealing since QHBs can host non-Abelian
anyons at $v_{T}<1$ [25, 28]. A recent work [38] discussed the possibility of
stabilizing multi-component Halperin’s $(m,m,m)$ state in a single layer
topological FB partially filled by spin-up and spin-down electrons, but this
system lacks the inter-component tunability present in QHBs.

In this Letter, we provide evidence for a direct analogue of QHB states in
bilayer Kagome lattices without magnetic field. Kagome lattice is the most
widely studied 2D lattice and material system hosting topological FBs [39, 40,
41, 42, 43, 44, 45, 46, 47]. Recently, it has generated a lot of experimental
interest with the realization of strongly correlated Kagome metals [48, 49,
50, 51, 52]. Importantly, bilayer Kagome lattice materials have also been
experimentally synthesized and known to be a host of quantum spin liquids [53,
54, 55]. In addition, a variety of topological FBs are proposed in multi-layer
heterostructures [56, 57], as well as in metal organic frameworks [58, 59, 60,
61]. The feasibility in synthesizing multi-layer FB lattices with FBs
belonging to individual layers, which can be treated similar to Landau levels
in QHB systems, motivates the investigation into its potential in realizing
QHB states without magnetic field. Here, we use exact diagonalization (ED) of
finite-size bilayer Kagome lattice and study characteristic features of QHB
states. Specifically, we show topological degeneracy of ground-state manifold
on a torus, spectral flow under flux insertion and excitonic off-diagonal
long-range order, indicating the stabilization of excitonic condensate states
at total filling $v_{T}=1$ and $v_{T}=1/3$ of FBs, and further illustrate the
potential of this lattice in stabilizing exotic tunable QHB states at
$v_{T}=2/3$, which potentially allows for the realization of non-Abelian
anyons without magnetic field.

We use a typical nearest-neighbor (NN) tight-binding model of bilayer Kagome
lattice (Fig. 1(a)), and consider the extreme limit of totally spin polarized
electrons in partially filed bands with spin-orbital coupling (SOC) that
conserves the out-of-plane spin-component [40], leading to kinetic part of
total Hamiltonian,

$\begin{split}H_{kin}=\sum_{\sigma}[-t&\sum_{<i,j>\alpha}c_{i\sigma\alpha}^{\dagger}c_{j\sigma\alpha}\\\
&+i\lambda\sum_{<i,j>\alpha\beta}\frac{2}{\sqrt{3}}(r_{ij}^{1}\times
r_{ij}^{2}).\sigma_{\alpha\beta}^{z}c_{i\alpha}^{\dagger}c_{j\beta}]\end{split}$
(1)

Figure 1: (a) Schematic of bilayer Kagome lattice with blue and orange colors
indicating atoms in layer L1 and L2 respectively. (b) Single-particle band
structure of bilayer Kagome lattice at zero tunneling.

where $t$ is the NN hopping integral, $\lambda$ represents SOC strength,
$r_{ij}^{1,2}$ are the two NN unit vectors, $\alpha(\beta)$ are spin indices,
and $\sigma(\sigma^{\prime})$ represents each layer of the bilayer lattice,
while $\sigma_{\alpha\beta}^{z}$ is the Pauli matrix. Interactions between
electrons are described by Hubbard model which includes NN intra-layer ($V$),
and direct inter-layer ($V_{\perp}$) interactions. $V_{\perp}$ depends on the
distance $d$ between the layers as shown in Fig. 1(a) which can be
independently varied, allowing for the tunability of inter-component
interactions. We also include tunneling between the layers ($t_{\perp}$) in
the interaction part of the Hamiltonian. This is closely related to the
experimental observation of excitonic condensate in bilayer graphene where a
tunneling barrier is introduced between the two layers to extend the lifetime
of electron-holes pairs [62, 63, 64, 65]. In essence, one can tune tunneling
between the layers independent of the layer distance. Here, we imagine a
similar tunneling barrier whose strength can be varied depending on the
tunneling material being used between the layers. It also implies that this
tunneling interaction makes the layer index a good quantum number although the
number of electrons in each layer individually is not conserved, in contrast
to introducing tunneling in $H_{kin}$ that would mix the layers while still
conserving the number of electrons in each band. Hence, the interaction part
of Hamiltonian in our model is given as,

$\begin{split}H_{int}=\sum_{\sigma}[V\sum_{<r,r^{\prime}>_{intra}}n_{r\sigma}n_{r^{\prime}\sigma}&+V_{\perp}\sum_{r}n_{r\sigma}n_{r\sigma^{\prime}}\\\
&+t_{\perp}\sum_{r}c_{r\sigma}^{\dagger}c_{r\sigma^{\prime}}]\end{split}$ (2)

where $n_{r\sigma}=c_{r\sigma}^{\dagger}c_{r\sigma}$ is the electron density
operator. The spin indices are suppressed. Throughout this work we set $V=2t$,
which is smaller than the single-particle gap separating the FBs (Fig. 1(b)),
while the distance dependence of $V_{\perp}$ is described as: $V_{\perp}=V$ if
$d\rightarrow 0$; $V_{\perp}=t$ if $d$ is finite.

Single-particle band structure is shown in Fig. 1(b). In the absence of
tunneling, the two FBs, both with non-zero and same Chern number [30], belong
to individual layers of the bilayer Kagome lattice, exactly matching the two
Landau levels in each layer of QHB. In our model, we use a slightly high SOC
($\lambda=0.3t$) to isolate the bottom FBs from other bands. This procedure is
widely used in realizing FCIs where a high flatness ratio of FBs is desirable.
Interestingly, such conditional parameters can be realized using Floquet band
engineering in real materials [66, 67, 47]. To study the effect of
interactions, we exactly diagonalize the full Hamiltonian
($H=H_{kin}+H_{int}$) in reciprocal space with interactions projected to the
two FBs of finite lattice containing total number of sites $N_{s}=6\times
N_{x}\times N_{y}$. Total filling factor is given by $v_{T}=N_{e}/(N_{x}\times
N_{y})$, where $N_{e}$ is the number of electrons in the system. Under
periodic boundary conditions (PBC), we implement translational symmetries and
diagonalize the Hamiltonian in each momentum sector $q=(2\pi k_{x}/N_{x},2\pi
k_{y}/N_{y})$ with $k_{x}$ and $k_{y}$ being the integers, labelled as
$(k_{x},k_{y})$. One can also assign a pseudospin notation to each FB and
define $S_{z}=(N_{\uparrow}-N_{\downarrow})/2$, where
$N_{\uparrow(\downarrow)}$ is the number of electrons in FB belonging to upper
(lower) layer. Complete methodology of ED for multiple bands is presented in
our previous work [6]].

First, we study the $v_{T}=1$ case. In QHBs with negligible $t_{\perp}$,
$v_{T}=1$ plateau manifests excitonic condensate with the wavefunction
described by Halperin’s $(1,1,1)$ state featuring a non-degenerate ground
state on two tori [38]. In Fig. 2(a) we show the energy spectra of $H$ for
$v_{T}=1$ with negligible $d$ and $t_{\perp}$ at multiple $S_{z}$. The
degeneracy of states with varying $S_{z}$ is a signature of spontaneous
symmetry breaking, also known as the “which layer” uncertainty for QHBs [20].
At a finite distance, the $S_{z}=0$ system or the case of balanced layers with
$N_{\uparrow}=N_{\downarrow}$, becomes energetically stable (Fig. S1 [68]). In
Fig. 2(b) we plot the energy spectra for $S_{z}=0$ case at a finite $d=0.3a$
and negligible $t_{\perp}$. There is a clear presence of a non-degenerate
ground state separated from excited states for two finite system sizes, which
also illustrates the convergence of our results. For all our subsequent
calculations we use the system size $4\times 3$. Halperin’s $(1,1,1)$ state on
two tori with PBC should belong to the total momentum sector
$(\sum_{i=1}^{N}k_{x}^{i},\sum_{i=1}^{N}k_{y}^{i})$, where $N=N_{x}\times
N_{y}$, implying all the reciprocal points are occupied; for a $4\times 3$
system size, it is the $(2,0)$ sector, as also shown in Fig. 2(b), implying a
non-trivial topological characteristic of the ground state.

Figure 2: ED calculation results at $v_{T}=1$. (a) Energy spectra of $H$ at
$d\rightarrow 0$ and $t_{\perp}=0$ with varying $S_{z}$ for $N_{s}=6\times
4\times 3$ system size. Red diamonds denote the ground states while yellow
denotes the excited states. (b) Momentum-resolved spectra of $H$ at $d=0.3a$,
$t_{\perp}=0$, and $S_{z}=0$ for two finite system sizes, $N_{s}=6\times
5\times 2$ and $N_{s}=6\times 4\times 3$ denoted by green circles and red
diamonds respectively. (c) Spectral flow of the system considered in (b) under
flux insertion in y-direction. (d) Eigenvalues of excitonic reduced two-body
density matrix for (b) plotted in descending order.

Beyond the energy spectra, we calculate spectral flow of this system under
twisted boundary condition in y-direction as plotted in Fig. 2(c). As is
evident in the figure, after insertion of one quantum flux states get back to
their initial configuration, and there is no mixing between the ground state
and excited states throughout the flux insertion. Thus, the ground state has a
total Chern number of 1, which also indicates a quantized Hall conductance of
$\sigma_{H}=e^{2}/h$ [35, 36]. An important feature of Halperin’s (1,1,1)
state is the excitonic condensation observed in experiments [20]. We
illustrate this by calculating eigenvalues of excitonic reduced two-body
density matrix [69],
$\rho^{(2)}(k,k^{\prime};\overline{k},\overline{k}^{\prime})=<\Psi|\psi_{2}^{\dagger(k)}\psi_{1}(k^{\prime})\psi_{1}^{\dagger}(\overline{k}^{\prime})\psi_{2}(\overline{k})|\Psi>$,
where $\psi_{1(2)}^{\dagger}(k)$ creates an electron in FB belonging to layer
$1(2)$ at reciprocal lattice point $k$, and $|\Psi>$ is the many-body ground
state wavefunction. As shown in Fig. 2(d), there is a presence of one large
eigenvalue which concretely illustrates off-diagonal long-range order of the
excitons formed between the occupied states in one FB and empty states in the
other [69].

In addition to the case of $v_{T}=1$, it is expected that similar excitonic
condensate state could be present at $v_{T}=1/3$ described by Halperin’s
$(3,3,3)$ wavefunction [38]. But so far, no experimental evidence for such a
state has been reported. In the case of bilayer Kagome, we next study the
characteristics of system at $v_{T}=1/3$ with negligible $d$ and $t_{\perp}$.
In Fig. 3(a), one clearly sees the signature of “which layer” uncertainty with
degenerate ground state manifold at multiple $S_{z}$. Different form
$v_{T}=1$, the ground state manifold in this case has a three-fold degeneracy
slightly lifted due to finite size effects. Moreover, as can be seen from Fig.
3(b), the three states belong to the momentum sectors $(2,0)$, $(2,1)$, and
$(2,2)$ respectively, as expected from generalized Pauli’s principle [33],
implying fractional statistics and non-trivial topology. In Fig. 3(c) we plot
the spectral flow of these states under flux insertion in y-direction. The
three states warp around each other and go back to their original
configuration after insertion of three flux quanta while the gap never closes.
This correlation indicates a fractional Hall conductance [35, 36]. We also
check for the exitonic off-diagonal long-range order by plotting eigenvalues
of two-body reduced density matrix in Fig. 3(d) and find the existence of one
large eigenvalue although not as prominent as for the case of $v_{T}=1$. Our
results here suggest that Halperin’s $(3,3,3)$ state featuring excitonic
condensate with fractional Hall conductance can in principle be realized in
bilayer Kagome system by only controlling the filling factor of electrons in
the system in contrast to tuning the magnetic field in QHBs.

Figure 3: ED calculation results at $v_{T}=1/3$. (a) Energy spectra of $H$ at
$d\rightarrow 0$, and $t_{\perp}=0$ with varying $S_{z}$ for $N_{s}=6\times
4\times 3$ system size. Red diamonds, blue and green triangles denote the
ground state manifold while yellow denotes the excited states. (b) Momentum-
resolved spectra of $H$ at $d=0$, $t_{\perp}=0$, and $S_{z}=0$. The three
quasi-degenerate ground states are denoted by red, blue and green colors
respectively. (c) Spectral flow of the system considered in (b) under flux
insertion in y-direction. (d) Eigenvalues of excitonic reduced two-body
density matrix for (b) plotted in descending order.

Next, we move on to illustrating the tunability of QHB states as realized in
this lattice system. Similar to QHBs, tunneling strength between the two
layers of Kagome lattice can be tuned by varying the tunneling material used
between the layers. At $v_{T}=2/3$ and finite $d=0.3a$, we show a tunneling
driven transition from Halperin’s $(3,3,0)$ state to the particle-hole
conjugate of Laughlin’s $1/3$ state for $S_{z}=0$, as depicted in Fig. 4(a).
Topological degeneracy of ground-state manifold on a torus is a powerful tool
that can distinguish the two phases. This was shown earlier by Haldane [70]
for single component FQH and later expanded by McDonald [71] for multi-
component FQH on a torus. Inter-component correlations in Halperin’s $(m,m,n)$
state can be studied using $K$-matrix, $K=\begin{bmatrix}m&n\\\
n&m\end{bmatrix}$, as formulated within the field theory framework [72, 73].
Topological degeneracy of ground state for a system with $v_{T}=p/q$ on a
torus is given by the determinant of $K$-matrix, $detK=qN^{\prime}$, where
$N^{\prime}$ is an integer describing the different translations of the
center-of-mass of different components, while $q$ determines the overall
center-of-mass degeneracy. Moreover, $N^{\prime}$ can be expressed as the sum
of $N_{0}$, the number of states at $(0,0)$, and $N_{B}$, number of states at
zone boundary momentum sectors $(0,\overline{N}/2)$, $(\overline{N}/2,0)$, and
$(\overline{N}/2,\overline{N}/2)$, where $\overline{N}=N_{e}/p$. For
homogenous liquids, $N^{\prime}=N_{0}+3N_{B}$ with two possibilities: if
$N_{e}/(N_{i}p)$ is even, $N_{0}=N_{B}=N^{\prime}/4$, otherwise,
$N_{0}=N^{\prime}$ and $N_{B}=0$, where $N_{i}$ is the number of electrons
with component $i$.

In the case of Halperin’s $(3,3,0)$ state at $v_{T}=2/3$, $detK=9$, $p=2$, and
$q=3$, implying $N^{\prime}=3$. For a finite system size $4\times 3$,
$N_{e}=8$, and since we are working with balanced layers ($S_{z}=0$),
$N_{i}=4$. Based on the above analysis, the 3 degenerate states corresponding
to $N^{\prime}=3$ should lie at $(0,0)$. Taking center-of-mass degeneracy into
account there should be additional 3-fold degeneracy for each of these states.
The center-of-mass degenerate points are given by Pauli’s generalized
principle [33] - if one state in the ground-state manifold lies in the
momentum sector $(k_{1},k_{2})$, the next state can be always found at
$(k_{1}+N_{e},k_{2}+N_{e})$ [modulo $(N_{x},N_{y})$], i.e., for
$(N_{x}=4,N_{y}=3)$ if there’re three degenerate states at $(0,0)$, there
should be three more states at $(0,1)$ and $(0,2)$ each. In Fig. 4(a), one can
see at small tunneling, the ground state manifold is composed of nine states,
exactly as predicted by the above analysis, confirming it is the Halperin’s
$(3,3,0)$ state. At strong tunneling, the bilayer effectively behaves as a
single layer due to strong correlations and stabilizes a particle-hole
conjugate of Laughlin’s $1/3$ state ($\overline{1/3}$) with only 3-fold
center-of-mass degeneracy at $(0,0)$, $(0,1)$, and $(0,2)$, which is also seen
in Fig. 4(a), similar to QHB system [28].

Figure 4: (a) Tunneling-driven phase transition at $v_{T}=2/3$, $d=0.3a$,
$S_{z}=0$ for $N_{s}=6\times 4\times 3$ system size. (b) Energy spectra of $H$
at $v_{T}=1/2$, $d=0.3a$, $t_{\perp}=0$, and $S_{z}=0$ for $N_{s}=6\times
4\times 3$ system size. Red diamonds denote the 8-fold degenerate ground
states while yellow denotes the excited states.

We finally do a similar analysis for the case of $v_{T}=1/2$ at a finite
$d=0.3a$, negligible $t_{\perp}$, and $S_{z}=0$. The ground state in this case
is described by Halperin’s $(3,3,1)$ wavefunction [25]. Here with $q=2$, and
$N^{\prime}=4$, the K-matrix analysis requires an 8-fold ground-state
degeneracy. The four states corresponding to $N^{\prime}=4$ belong to sectors
$(0,0)$, $(0,\overline{N}/2=3)$, $(\overline{N}/2=3,\overline{N}/2=3)$, and
$(\overline{N}/2=3,0)$. Since $N_{x}=4$, and $N_{y}=3$, this corresponds to
two states at $(0,0)$ and $(3,0)$ each, in addition to $q=2$ center-of-mass
degeneracy. The 8-fold degenerate ground states, hence, should belong to the
$(0,0)$, $(1,0)$, $(2,0)$, and $(3,0)$ momentum sectors, each two-fold
degenerate. In Fig. 4(b), we plot the energy spectrum for this lattice with
8-fold degenerate ground-state manifold exactly matching the $K$-matrix
analysis. Importantly, in this system a tunneling driven transition to non-
Abelian Moore-Read state is proposed whose investigation, however, requires
numerical calculations with much larger system sizes as shown earlier for QHBs
[25], which has been left for future work.

In conclusion, we have demonstrated the potential of bilayer Kagome lattice,
as a prototypical example of experimentally feasible multi-layer FB lattices,
to realize anomalous quantum Hall bilayer effect. In addition to the
stabilization of excitonic condensate Halperin’s $(1,1,1)$ and $(3,3,3)$
states, we demonstrate striking tunability of QHB states with interlayer
tunneling strength which could lead to the realization of non-Abelian anyonic
states without magnetic field. Interestingly, a non-Abelian interlayer
Pfaffian state with 9-fold ground degeneracy is proposed for $v_{T}=2/3$ QHB
when interlayer interaction becomes slightly attractive [28]. We see a similar
9-fold degeneracy of ground state manifold at negative interlayer interaction
strength in our calculations (Fig. S2 [68]) but more concrete evidence
requires the analysis of orbital-cut entanglement spectra demanding further
investigation in this direction. With our work, we hence provide a new
platform to study anyonic statistics and further elaborate the role of unique
topological properties of FBs in realizing QHB states without magnetic field.

This work is supported by US Department of Energy-Basic Energy Sciences (Grant
No. DE-FG02- 04ER46148). All calculations were done on the CHPC at the
University of Utah.

## Supplementary Material

Figure S1: Energy spectra of $H$ at $v_{T}=1$, $d=0.3a$, and $t_{\perp}=0$
with varying $S_{z}$ for $N_{s}=6\times 4\times 3$ system size. Red diamonds
denote the ground states while yellow denotes the excited states. Figure S2:
Energy spectra of $H$ at $v_{T}=2/3$, $d=0.3a$, $t_{\perp}=0$, and $S_{z}=0$
for $N_{s}=6\times 4\times 3$ system size with attractive interlayer
interaction. Red diamonds denote the 9-fold degenerate ground states while
yellow denotes the excited states.

## References

  * Tsui  _et al._ [1982] D. C. Tsui, H. L. Stormer, and A. C. Gossard, Two-dimensional magnetotransport in the extreme quantum limit, Physical Review Letters 48, 1559 (1982).
  * Laughlin [1983] R. B. Laughlin, Anomalous quantum hall effect: an incompressible quantum fluid with fractionally charged excitations, Physical Review Letters 50, 1395 (1983).
  * Haldane [1983] F. D. M. Haldane, Fractional quantization of the hall effect: a hierarchy of incompressible quantum fluid states, Physical Review Letters 51, 605 (1983).
  * Halperin [1984] B. I. Halperin, Statistics of quasiparticles and the hierarchy of fractional quantized hall states, Physical Review Letters 52, 1583 (1984).
  * Arovas  _et al._ [1984] D. Arovas, J. R. Schrieffer, and F. Wilczek, Fractional statistics and the quantum hall effect, Physical review letters 53, 722 (1984).
  * Moore and Read [1991] G. Moore and N. Read, Non-Abelions in the fractional quantum hall effect, Nuclear Physics B 360, 362 (1991).
  * Nayak  _et al._ [2008] C. Nayak, S. H. Simon, A. Stern, M. Freedman, and S. D. Sarma, Non-Abelian anyons and topological quantum computation, Reviews of Modern Physics 80, 1083 (2008).
  * Greiter  _et al._ [1991] M. Greiter, X.-G. Wen, and F. Wilczek, Paired hall state at half filling, Physical review letters 66, 3205 (1991).
  * Read and Rezayi [1999] N. Read and E. Rezayi, Beyond paired quantum hall states: Parafermions and incompressible states in the first excited landau level, Physical Review B 59, 8084 (1999).
  * Rezayi and Read [2009] E. Rezayi and N. Read, Non-Abelian quantized hall states of electrons at filling factors 12/5 and 13/5 in the first excited landau level, Physical Review B 79, 075306 (2009).
  * Halperin [1983] B. I. Halperin, Theory of the quantized hall conductance, Helvetica Physica Acta 56, 75 (1983).
  * Haldane and Rezayi [1988] F. Haldane and E. Rezayi, Spin-singlet wave function for the half-integral quantum hall effect, Physical review letters 60, 956 (1988).
  * Wen [2000] X.-G. Wen, Continuous topological phase transitions between clean quantum hall states, Physical review letters 84, 3950 (2000).
  * Barkeshli and Wen [2011] M. Barkeshli and X.-G. Wen, Bilayer quantum hall phase transitions and the orbifold non-abelian fractional quantum hall states, Physical Review B 84, 115121 (2011).
  * Suen  _et al._ [1992] Y. Suen, L. Engel, M. Santos, M. Shayegan, and D. Tsui, Observation of a $\nu=1/2$ fractional quantum hall state in a double-layer electron system, Physical review letters 68, 1379 (1992).
  * Eisenstein  _et al._ [1992] J. Eisenstein, G. Boebinger, L. Pfeiffer, K. West, and S. He, New fractional quantum hall state in double-layer two-dimensional electron systems, Physical review letters 68, 1383 (1992).
  * Suen  _et al._ [1994] Y. Suen, H. Manoharan, X. Ying, M. Santos, and M. Shayegan, Origin of the $\nu=1/2$ fractional quantum hall state in wide single quantum wells, Physical review letters 72, 3405 (1994).
  * Shabani  _et al._ [2013] J. Shabani, Y. Liu, M. Shayegan, L. Pfeiffer, K. West, and K. Baldwin, Phase diagrams for the stability of the $\nu=1/2$ fractional quantum hall effect in electron systems confined to symmetric, wide GaAs quantum wells, Physical Review B 88, 245413 (2013).
  * Eisenstein and MacDonald [2004] J. Eisenstein and A. MacDonald, Bose-Einstein condensation of excitons in bilayer electron systems, Nature 432, 691 (2004).
  * Eisenstein [2014] J. Eisenstein, Exciton condensation in bilayer quantum hall systems, Annu. Rev. Condens. Matter Phys. 5, 159 (2014).
  * Eisenstein  _et al._ [2019] J. Eisenstein, L. Pfeiffer, and K. West, Precursors to exciton condensation in quantum hall bilayers, Physical Review Letters 123, 066802 (2019).
  * Sheng  _et al._ [2003] D. Sheng, L. Balents, and Z. Wang, Phase diagram for quantum hall bilayers at $\nu=1$, Physical review letters 91, 116802 (2003).
  * Simon  _et al._ [2003] S. H. Simon, E. Rezayi, and M. V. Milovanovic, Coexistence of composite bosons and composite fermions in $\nu=1/2+1/2$ quantum hall bilayers, Physical review letters 91, 046803 (2003).
  * Möller  _et al._ [2008] G. Möller, S. H. Simon, and E. H. Rezayi, Paired composite fermion phase of quantum hall bilayers at $\nu=1/2+1/2$, Physical review letters 101, 176803 (2008).
  * Zhu  _et al._ [2016] W. Zhu, Z. Liu, F. Haldane, and D. Sheng, Fractional quantum hall bilayers at half filling: Tunneling-driven non-Abelian phase, Physical Review B 94, 245147 (2016).
  * McDonald and Haldane [1996] I. McDonald and F. Haldane, Topological phase transition in the $\nu=2/3$ quantum hall effect, Physical Review B 53, 15845 (1996).
  * Vaezi and Barkeshli [2014] A. Vaezi and M. Barkeshli, Fibonacci anyons from abelian bilayer quantum hall states, Physical review letters 113, 236804 (2014).
  * Geraedts  _et al._ [2015] S. Geraedts, M. P. Zaletel, Z. Papić, and R. S. Mong, Competing abelian and non-abelian topological orders in $\nu=1/3+1/3$ quantum hall bilayers, Physical Review B 91, 205139 (2015).
  * Bergholtz and Liu [2013] E. J. Bergholtz and Z. Liu, Topological flat band models and fractional chern insulators, International Journal of Modern Physics B 27, 1330017 (2013).
  * Liu  _et al._ [2014] Z. Liu, F. Liu, and Y.-S. Wu, Exotic electronic states in the world of flat bands: From theory to material, Chinese Physics B 23, 077308 (2014).
  * Li  _et al._ [2014] W. Li, Z. Liu, Y.-S. Wu, and Y. Chen, Exotic fractional topological states in a two-dimensional organometallic material, Physical Review B 89, 125411 (2014).
  * Sheng  _et al._ [2011] D. Sheng, Z.-C. Gu, K. Sun, and L. Sheng, Fractional quantum hall effect in the absence of Landau levels, Nature communications 2, 1 (2011).
  * Regnault and Bernevig [2011] N. Regnault and B. A. Bernevig, Fractional chern insulator, Physical Review X 1, 021014 (2011).
  * Wu  _et al._ [2012] Y.-L. Wu, B. A. Bernevig, and N. Regnault, Zoology of fractional chern insulators, Physical Review B 85, 075116 (2012).
  * Laughlin [1981] R. B. Laughlin, Quantized hall conductivity in two dimensions, Physical Review B 23, 5632 (1981).
  * Niu  _et al._ [1985] Q. Niu, D. J. Thouless, and Y.-S. Wu, Quantized hall conductance as a topological invariant, Physical Review B 31, 3372 (1985).
  * Sterdyniak  _et al._ [2011] A. Sterdyniak, N. Regnault, and B. A. Bernevig, Extracting excitations from model state entanglement, Physical review letters 106, 100405 (2011).
  * Zeng  _et al._ [2020] T.-S. Zeng, D. Sheng, and W. Zhu, Quantum hall effects of exciton condensate in topological flat bands, Physical Review B 101, 195310 (2020).
  * Mielke [1992] A. Mielke, Exact ground states for the hubbard model on the kagome lattice, Journal of Physics A: Mathematical and General 25, 4335 (1992).
  * Tang  _et al._ [2011] E. Tang, J.-W. Mei, and X.-G. Wen, High-temperature fractional quantum hall states, Physical review letters 106, 236802 (2011).
  * Liu  _et al._ [2009] G. Liu, P. Zhang, Z. Wang, and S.-S. Li, Spin hall effect on the kagome lattice with Rashba spin-orbit interaction, Physical Review B 79, 035323 (2009).
  * Guo and Franz [2009] H.-M. Guo and M. Franz, Topological insulator on the kagome lattice, Physical Review B 80, 113102 (2009).
  * Yan  _et al._ [2011] S. Yan, D. A. Huse, and S. R. White, Spin-liquid ground state of the S=1/2 kagome Heisenberg antiferromagnet, Science 332, 1173 (2011).
  * Wang and Zhang [2010] Z. Wang and P. Zhang, Quantum spin hall effect and spin-charge separation in a kagomé lattice, New Journal of Physics 12, 043055 (2010).
  * Zhou  _et al._ [2022] Y. Zhou, G. Sethi, H. Liu, Z. Wang, and F. Liu, Excited quantum anomalous and spin Hall effect: dissociation of flat-bands-enabled excitonic insulator state, Nanotechnology 33, 415001 (2022).
  * Sethi  _et al._ [2021] G. Sethi, Y. Zhou, L. Zhu, L. Yang, and F. Liu, Flat-band-enabled triplet excitonic insulator in a diatomic kagome lattice, Physical Review Letters 126, 196403 (2021).
  * Liu  _et al._ [2022a] H. Liu, G. Sethi, D. Sheng, Y. Zhou, J.-T. Sun, S. Meng, and F. Liu, High-temperature fractional quantum hall state in the floquet kagome flat band, Physical Review B 105, L161108 (2022a).
  * Kang  _et al._ [2020] M. Kang, L. Ye, S. Fang, J.-S. You, A. Levitan, M. Han, J. I. Facio, C. Jozwiak, A. Bostwick, E. Rotenberg, _et al._ , Dirac fermions and flat bands in the ideal kagome metal FeSn, Nature materials 19, 163 (2020).
  * Ye  _et al._ [2018] L. Ye, M. Kang, J. Liu, F. Von Cube, C. R. Wicker, T. Suzuki, C. Jozwiak, A. Bostwick, E. Rotenberg, D. C. Bell, _et al._ , Massive dirac fermions in a ferromagnetic kagome metal, Nature 555, 638 (2018).
  * Tan  _et al._ [2021] H. Tan, Y. Liu, Z. Wang, and B. Yan, Charge density waves and electronic properties of superconducting kagome metals, Physical review letters 127, 046401 (2021).
  * Liu  _et al._ [2022b] G. Liu, X. Ma, K. He, Q. Li, H. Tan, Y. Liu, J. Xu, W. Tang, K. Watanabe, T. Taniguchi, _et al._ , Observation of anomalous amplitude modes in the kagome metal CsV3Sb5, Nature communications 13, 1 (2022b).
  * Zhao  _et al._ [2021] H. Zhao, H. Li, B. R. Ortiz, S. M. Teicher, T. Park, M. Ye, Z. Wang, L. Balents, S. D. Wilson, and I. Zeljkovic, Cascade of correlated electron states in the kagome superconductor CsV3Sb5, Nature 599, 216 (2021).
  * Ni  _et al._ [2018] J. Ni, Q. Liu, Y. Yu, E. Cheng, Y. Huang, Z. Liu, X. Wang, Y. Sui, and S. Li, Ultralow-temperature heat transport in the quantum spin liquid candidate Ca10Cr7O28 with a bilayer kagome lattice, Physical Review B 97, 104413 (2018).
  * Balz  _et al._ [2016] C. Balz, B. Lake, J. Reuther, H. Luetkens, R. Schönemann, T. Herrmannsdörfer, Y. Singh, A. Nazmul Islam, E. M. Wheeler, J. A. Rodriguez-Rivera, _et al._ , Physical realization of a quantum spin liquid based on a complex frustration mechanism, Nature Physics 12, 942 (2016).
  * Balodhi and Singh [2017] A. Balodhi and Y. Singh, Synthesis and pressure and field-dependent magnetic properties of the kagome-bilayer spin liquid Ca10Cr7O28, Physical Review Materials 1, 024407 (2017).
  * Balents  _et al._ [2020] L. Balents, C. R. Dean, D. K. Efetov, and A. F. Young, Superconductivity and strong correlations in moiré flat bands, Nature Physics 16, 725 (2020).
  * Zhang  _et al._ [2019] Y.-H. Zhang, D. Mao, Y. Cao, P. Jarillo-Herrero, and T. Senthil, Nearly flat chern bands in moiré superlattices, Physical Review B 99, 075127 (2019).
  * Kambe  _et al._ [2013] T. Kambe, R. Sakamoto, K. Hoshiko, K. Takada, M. Miyachi, J.-H. Ryu, S. Sasaki, J. Kim, K. Nakazato, M. Takata, _et al._ , $\pi$-Conjugated nickel bis (dithiolene) complex nanosheet, Journal of the American Chemical Society 135, 2462 (2013).
  * Jiang  _et al._ [2021] W. Jiang, X. Ni, and F. Liu, Exotic topological bands and quantum states in metal-organic and covalent-organic frameworks, Accounts of Chemical Research 54, 416 (2021).
  * Ni  _et al._ [2022] X. Ni, H. Li, F. Liu, and J.-L. Brédas, Engineering of flat bands and dirac bands in two-dimensional covalent organic frameworks (COFs): relationships among molecular orbital symmetry, lattice symmetry, and electronic-structure characteristics, Materials Horizons 9, 88 (2022).
  * Zhang  _et al._ [2016] X. Zhang, Z. Wang, M. Zhao, and F. Liu, Tunable topological states in electron-doped HTT-Pt, Physical Review B 93, 165401 (2016).
  * Kharitonov and Efetov [2008] M. Y. Kharitonov and K. B. Efetov, Electron screening and excitonic condensation in double-layer graphene systems, Physical Review B 78, 241401 (2008).
  * Li  _et al._ [2017] J. Li, T. Taniguchi, K. Watanabe, J. Hone, and C. Dean, Excitonic superfluid phase in double bilayer graphene, Nature Physics 13, 751 (2017).
  * Liu  _et al._ [2017] X. Liu, K. Watanabe, T. Taniguchi, B. I. Halperin, and P. Kim, Quantum hall drag of exciton condensate in graphene, Nature Physics 13, 746 (2017).
  * Fuhrer and Hamilton [2016] M. S. Fuhrer and A. R. Hamilton, Chasing the exciton condensate, Physics 9, 80 (2016).
  * Ge and Kolodrubetz [2021] R.-C. Ge and M. Kolodrubetz, Floquet engineering flat bands for bosonic fractional quantum hall with superconducting circuits, Physical Review B 104, 035427 (2021).
  * Grushin  _et al._ [2014] A. G. Grushin, Á. Gómez-León, and T. Neupert, Floquet fractional chern insulators, Physical review letters 112, 156801 (2014).
  * [68] See supplementary material at for ED results on finite distance bilayer Kagome at $v_{T}=1$ with varying $S_{z}$, and possible non-Abelian phase $v_{T}=2/3$.
  * Sethi  _et al._ [2022] G. Sethi, M. Cuma, and F. Liu, Excitonic condensate in flat valence and conduction bands of opposite chirality, arXiv preprint arXiv:2210.03252 (2022).
  * Haldane [1985] F. Haldane, Many-particle translational symmetries of two-dimensional electrons at rational landau-level filling, Physical review letters 55, 2095 (1985).
  * McDonald [1995] I. McDonald, Degeneracy of multicomponent quantum hall states satisfying periodic boundary conditions, Physical Review B 51, 10851 (1995).
  * Wen and Zee [1992] X.-G. Wen and A. Zee, Classification of Abelian quantum hall states and matrix formulation of topological fluids, Physical Review B 46, 2290 (1992).
  * Blok and Wen [1990] B. Blok and X.-G. Wen, Effective theories of the fractional quantum hall effect at generic filling fractions, Physical Review B 42, 8133 (1990).