an active field aiming at enriching textual representations with visual
information, which has been shown to improve performance on a variety of core
NLP tasks (Bruni et al., 2014; Baroni, 2016; Kiela, 2017). Some recent work
also propose unified-modal models for language understanding and generation
UNIMO (Li et al., 2020c). Learning grounded language representation is a
promising direction for improving the expressiveness of NLG models, thus
improving the faithfulness of their generated texts.

##### Incorporating Causal Inference

Existing language models are mostly correlation prediction models, where
predictions are due to correlation rather than causal inference. Therefore,
these correlational predictive models are not credible and may lead to errors
in out-of-distribution or long-tailed situations. Clearly, the lack of
causality is one of the main reasons for poor model generalization and
faithfulness. Incorporating causal inference into language model may be the
fundamental method to solve the faithfulness of NLG models.

##  6 Conclusion

In this survey, we conduct a systematic overview of the faithfulness problem
across different NLG tasks. We organize and discuss the faithfulness analysis,
evaluation metrics and optimization methods in a combined manner under a
general categorization standard. The evaluation metrics for different NLG
tasks are categorized into four types: Entailment-based, QA-based, Fact-based
and Others. The optimization methods are categorized into six types: Factual
Guidance, Auxiliary Task, Post-Editing, Learning Method, Constrained Decoding
and Others. All the evaluation metrics and optimization methods are discussed
and compared to facilitate both task-specific and task-agnostics understanding
of the faithfulness problem. In addition, we propose potential future
directions according to the challenges of this problem and the current
research status of evaluation metrics and optimization methods.

## References

  * Allahyari et al. (2017) Mehdi Allahyari, Seyedamin Pouriyeh, Mehdi Assefi, Saeid Safaei, Elizabeth D Trippe, Juan B Gutierrez, and Krys Kochut.  Text summarization techniques: a brief survey.  _arXiv preprint arXiv:1707.02268_ , 2017.
  * Anderson et al. (2017) Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould.  Guided Open Vocabulary Image Captioning with Constrained Beam Search.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 936–945, Copenhagen, Denmark, 2017. Association for Computational Linguistics.  doi: 10.18653/v1/D17-1098.  URL https://aclanthology.org/D17-1098.
  * Aralikatte et al. (2021) Rahul Aralikatte, Shashi Narayan, Joshua Maynez, Sascha Rothe, and Ryan T. McDonald.  Focus attention: Promoting faithfulness and diversity in summarization.  In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , pages 6078–6095. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.acl-long.474.  URL https://doi.org/10.18653/v1/2021.acl-long.474.
  * Arun et al. (2020) Ankit Arun, Soumya Batra, Vikas Bhardwaj, Ashwini Challa, Pinar Donmez, Peyman Heidari, Hakan Inan, Shashank Jain, Anuj Kumar, Shawn Mei, Karthik Mohan, and Michael White.  Best practices for data-efficient modeling in NLG: how to train production-ready neural models with less data.  In Ann Clifton and Courtney Napoles, editors, _Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020 \- Industry Track, Online, December 12, 2020_ , pages 64–77. International Committee on Computational Linguistics, 2020.  doi: 10.18653/v1/2020.coling-industry.7.  URL https://doi.org/10.18653/v1/2020.coling-industry.7.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_ , 2014.
  * Balakrishnan et al. (2019) Anusha Balakrishnan, Jinfeng Rao, Kartikeya Upasani, Michael White, and Rajen Subba.  Constrained decoding for neural NLG from compositional representations in task-oriented dialogue.  In Anna Korhonen, David R. Traum, and Lluís Màrquez, editors, _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 831–844. Association for Computational Linguistics, 2019.  doi: 10.18653/v1/p19-1080.  URL https://doi.org/10.18653/v1/p19-1080.
  * Banerjee and Lavie (2005) Satanjeev Banerjee and Alon Lavie.  Meteor: An automatic metric for mt evaluation with improved correlation with human judgments.  In _Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization_ , pages 65–72, 2005.
  * Bao et al. (2020) Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang.  PLATO: pre-trained dialogue generation model with discrete latent variable.  In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 85–96. Association for Computational Linguistics, 2020.  doi: 10.18653/v1/2020.acl-main.9.  URL https://doi.org/10.18653/v1/2020.acl-main.9.
  * Baroni (2016) Marco Baroni.  Grounding distributional semantics in the visual world.  _Language and Linguistics Compass_ , 10(1):3–13, 2016.
  * Barrantes et al. (2020) Mario Barrantes, Benedikt Herudek, and Richard Wang.  Adversarial NLI for factual correctness in text summarisation models.  _CoRR_ , abs/2005.11739, 2020.  URL https://arxiv.org/abs/2005.11739.
  * Belinkov and Bisk (2017) Yonatan Belinkov and Yonatan Bisk.  Synthetic and natural noise both break neural machine translation.  _arXiv preprint arXiv:1711.02173_ , 2017.
  * Boag et al. (2020) William Boag, Tzu-Ming Harry Hsu, Matthew McDermott, Gabriela Berner, Emily Alesentzer, and Peter Szolovits.  Baselines for chest x-ray report generation.  In _Machine Learning for Health Workshop_ , pages 126–140. PMLR, 2020\.
  * Brown et al. (1983) Gillian Brown, Gillian D Brown, Gillian R Brown, George Yule, and Brown Gillian.  _Discourse analysis_.  Cambridge university press, 1983.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Bruni et al. (2014) Elia Bruni, Nam-Khanh Tran, and Marco Baroni.  Multimodal distributional semantics.  _Journal of artificial intelligence research_ , 49:1–47, 2014.
  * Cao et al. (2020) Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung.  Factual error correction for abstractive summarization models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6251–6258, 2020.
  * Cao et al. (2021) Meng Cao, Yue Dong, and Jackie Chi Kit Cheung.  Inspecting the Factuality of Hallucinated Entities in Abstractive Summarization.  _arXiv:2109.09784 [cs]_ , August 2021.  URL http://arxiv.org/abs/2109.09784.  arXiv: 2109.09784.
  * Cao and Wang (2021) Shuyang Cao and Lu Wang.  CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6633–6649, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.  URL https://aclanthology.org/2021.emnlp-main.532.
  * Cao et al. (2017) Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li.  Faithful to the Original: Fact Aware Neural Abstractive Summarization.  _arXiv:1711.04434 [cs]_ , November 2017.  URL http://arxiv.org/abs/1711.04434.  arXiv: 1711.04434.
  * Cao et al. (2018) Ziqiang Cao, Wenjie Li, Sujian Li, and Furu Wei.  Retrieve, rerank and rewrite: Soft template based neural summarization.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 152–161, 2018.
  * Chen et al. (2020a) Anthony Chen, Gabriel Stanovsky, Sameer Singh, and Matt Gardner.  MOCHA: A dataset for training and evaluating generative reading comprehension metrics.  In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_ , pages 6521–6532. Association for Computational Linguistics, 2020a.  doi: 10.18653/v1/2020.emnlp-main.528.  URL https://doi.org/10.18653/v1/2020.emnlp-main.528.
  * Chen et al. (2017) Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen.  Enhanced LSTM for natural language inference.  In Regina Barzilay and Min-Yen Kan, editors, _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers_ , pages 1657–1668. Association for Computational Linguistics, 2017.  doi: 10.18653/v1/P17-1152.  URL https://doi.org/10.18653/v1/P17-1152.
  * Chen et al. (2021) Sihao Chen, Fan Zhang, Kazoo Sone, and Dan Roth.  Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5935–5941, Online, June 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.naacl-main.475.  URL https://aclanthology.org/2021.naacl-main.475.
  * Chen et al. (2020b) Wenhu Chen, Yu Su, Xifeng Yan, and William Yang Wang.  Kgpt: Knowledge-grounded pre-training for data-to-text generation.  _arXiv preprint arXiv:2010.02307_ , 2020b.
  * Chen et al. (2020c) Zhihong Chen, Yan Song, Tsung-Hui Chang, and Xiang Wan.  Generating radiology reports via memory-driven transformer.  _arXiv preprint arXiv:2010.16056_ , 2020c.
  * Cheng et al. (2018) Yong Cheng, Zhaopeng Tu, Fandong Meng, Junjie Zhai, and Yang Liu.  Towards robust neural machine translation.  _arXiv preprint arXiv:1805.06130_ , 2018.
  * Cheng et al. (2019) Yong Cheng, Lu Jiang, and Wolfgang Macherey.  Robust Neural Machine Translation with Doubly Adversarial Inputs.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4324–4333, Florence, Italy, July 2019. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1425.  URL https://aclanthology.org/P19-1425.
  * Choubey et al. (2021) Prafulla Kumar Choubey, Jesse Vig, Wenhao Liu, and Nazneen Fatema Rajani.  MoFE: Mixture of Factual Experts for Controlling Hallucinations in Abstractive Summarization.  _arXiv:2110.07166 [cs]_ , October 2021.  URL http://arxiv.org/abs/2110.07166.  arXiv: 2110.07166.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning.  ELECTRA: pre-training text encoders as discriminators rather than generators.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.  URL https://openreview.net/forum?id=r1xMH1BtvB.
  * Demszky et al. (2018) Dorottya Demszky, Kelvin Guu, and Percy Liang.  Transforming question answering datasets into natural language inference datasets.  _CoRR_ , abs/1809.02922, 2018.  URL http://arxiv.org/abs/1809.02922.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Dhingra et al. (2019) Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-Wei Chang, Dipanjan Das, and William Cohen.  Handling Divergent Reference Texts when Evaluating Table-to-Text Generation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4884–4895, Florence, Italy, 2019. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1483.  URL https://aclanthology.org/P19-1483.
  * Dinan et al. (2019) Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston.  Wizard of wikipedia: Knowledge-powered conversational agents.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net, 2019.  URL https://openreview.net/forum?id=r1l73iRqKm.
  * Dong et al. (2019) Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon.  Unified language model pre-training for natural language understanding and generation.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Dong et al. (2020) Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, and Jingjing Liu.  Multi-fact correction in abstractive text summarization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9320–9331, 2020.
  * Dou et al. (2021) Zi-Yi Dou, Pengfei Liu, Hiroaki Hayashi, Zhengbao Jiang, and Graham Neubig.  Gsum: A general framework for guided neural abstractive summarization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4830–4842, 2021.
  * Du et al. (2017) Xinya Du, Junru Shao, and Claire Cardie.  Learning to ask: Neural question generation for reading comprehension.  _arXiv preprint arXiv:1705.00106_ , 2017.
  * Durmus et al. (2020) Esin Durmus, He He, and Mona Diab.  FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization.  _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5055–5070, 2020.  doi: 10.18653/v1/2020.acl-main.454.  URL http://arxiv.org/abs/2005.03754.  arXiv: 2005.03754.
  * Dziri et al. (2021a) Nouha Dziri, Andrea Madotto, Osmar Zaïane, and Avishek Joey Bose.  Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 2197–2214, Online and Punta Cana, Dominican Republic, November 2021a. Association for Computational Linguistics.  URL https://aclanthology.org/2021.emnlp-main.168.
  * Dziri et al. (2021b) Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter.  Evaluating groundedness in dialogue systems: The BEGIN benchmark.  _CoRR_ , abs/2105.00071, 2021b.  URL https://arxiv.org/abs/2105.00071.
  * Ebrahimi et al. (2017) Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou.  Hotflip: White-box adversarial examples for text classification.  _arXiv preprint arXiv:1712.06751_ , 2017.
  * Egan et al. (2021) Nicholas Egan, Oleg Vasilyev, and John Bohannon.  Play the Shannon Game With Language Models: A Human-Free Approach to Summary Evaluation.  _arXiv:2103.10918 [cs]_ , December 2021.  URL http://arxiv.org/abs/2103.10918.  arXiv: 2103.10918.
  * Fabbri et al. (2021a) Alexander R. Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong.  QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization.  _arXiv:2112.08542 [cs]_ , December 2021a.  URL http://arxiv.org/abs/2112.08542.  arXiv: 2112.08542.
  * Fabbri et al. (2021b) Alexander R. Fabbri, Xiaojian Wu, Srini Iyer, Haoran Li, and Mona Diab.  AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization.  _arXiv:2111.06474 [cs]_ , November 2021b.  URL http://arxiv.org/abs/2111.06474.  arXiv: 2111.06474.
  * Falke et al. (2019) Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych.  Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2214–2220, Florence, Italy, July 2019. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1213.  URL https://aclanthology.org/P19-1213.
  * Feng et al. (2020) Yang Feng, Wanying Xie, Shuhao Gu, Chenze Shao, Wen Zhang, Zhengxin Yang, and Dong Yu.  Modeling fluency and faithfulness for diverse neural machine translation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 59–66, 2020.
  * Filippova (2020) Katja Filippova.  Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 864–870, Online, 2020. Association for Computational Linguistics.  doi: 10.18653/v1/2020.findings-emnlp.76.
  * Fillmore et al. (1976) Charles J Fillmore et al.  Frame semantics and the nature of language.  In _Annals of the New York Academy of Sciences: Conference on the origin and development of language and speech_ , volume 280, pages 20–32. New York, 1976.
  * Fleiss (1971) Joseph L Fleiss.  Measuring nominal scale agreement among many raters.  _Psychological bulletin_ , 76(5):378, 1971.
  * Frank and Goodman (2012) Michael C Frank and Noah D Goodman.  Predicting pragmatic reasoning in language games.  _Science_ , 336(6084):998–998, 2012.
  * Gabriel et al. (2021a) Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, and Yejin Choi.  Discourse understanding and factual consistency in abstractive summarization.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 435–447, 2021a.
  * Gabriel et al. (2021b) Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, and Jianfeng Gao.  GO FIGURE: A Meta Evaluation of Factuality in Summarization.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 478–487, Online, August 2021b. Association for Computational Linguistics.  doi: 10.18653/v1/2021.findings-acl.42.  URL https://aclanthology.org/2021.findings-acl.42.
  * Gao et al. (2019) Xiang Gao, Yizhe Zhang, Sungjin Lee, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan.  Structuring latent spaces for stylized response generation.  In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_ , pages 1814–1823. Association for Computational Linguistics, 2019.  doi: 10.18653/v1/D19-1190.  URL https://doi.org/10.18653/v1/D19-1190.
  * Garg et al. (2019) Sarthak Garg, Stephan Peitz, Udhyakumar Nallasamy, and Matthias Paulik.  Jointly learning to align and translate with transformer models.  _arXiv preprint arXiv:1909.02074_ , 2019.
  * Gatt and Krahmer (2018) Albert Gatt and Emiel Krahmer.  Survey of the state of the art in natural language generation: Core tasks, applications and evaluation.  _Journal of Artificial Intelligence Research_ , 61:65–170, 2018.
  * Gehrmann et al. (2021) Sebastian Gehrmann, Tosin P. Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Aremu Anuoluwapo, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh D. Dhole, Wanyu Du, Esin Durmus, Ondrej Dusek, Chris Emezue, Varun Gangal, Cristina Garbacea, Tatsunori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Dhruv Kumar, Faisal Ladhak, Aman Madaan, Mounica Maddela, Khyati Mahajan, Saad Mahamood, Bodhisattwa Prasad Majumder, Pedro Henrique Martins, Angelina McMillan-Major, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Rubungo Andre Niyongabo, Salomey Osei, Ankur P. Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanam, João Sedoc, Thibault Sellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, and Jiawei Zhou.  The GEM benchmark: Natural language generation, its evaluation and metrics.  _CoRR_ , abs/2102.01672, 2021.  URL https://arxiv.org/abs/2102.01672.
  * Ghazvininejad et al. (2018) Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley.  A knowledge-grounded neural conversation model.  In Sheila A. McIlraith and Kilian Q. Weinberger, editors, _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_ , pages 5110–5117. AAAI Press, 2018.  URL https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16710.
  * Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  _arXiv preprint arXiv:1412.6572_ , 2014.
  * Goodrich et al. (2019) Ben Goodrich, Vinay Rao, Mohammad Saleh, and Peter J. Liu.  Assessing The Factual Accuracy of Generated Text.  _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 166–175, July 2019.  doi: 10.1145/3292500.3330955.  URL http://arxiv.org/abs/1905.13322.  arXiv: 1905.13322.
  * Goyal and Durrett (2020) Tanya Goyal and Greg Durrett.  Evaluating Factuality in Generation with Dependency-level Entailment.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3592–3603, Online, November 2020. Association for Computational Linguistics.  doi: 10.18653/v1/2020.findings-emnlp.322.  URL https://aclanthology.org/2020.findings-emnlp.322.
  * Goyal and Durrett (2021) Tanya Goyal and Greg Durrett.  Annotating and Modeling Fine-grained Factuality in Summarization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1449–1462, Online, June 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.naacl-main.114.  URL https://aclanthology.org/2021.naacl-main.114.
  * Graves (2013) Alex Graves.  Generating sequences with recurrent neural networks.  _arXiv preprint arXiv:1308.0850_ , 2013.
  * Gu et al. (2016) Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li.  Incorporating copying mechanism in sequence-to-sequence learning.  _arXiv preprint arXiv:1603.06393_ , 2016.
  * Gupta et al. (2021) Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong.  Dialfact: A benchmark for fact-checking in dialogue.  _CoRR_ , abs/2110.08222, 2021.  URL https://arxiv.org/abs/2110.08222.
  * Hadash et al. (2018) Guy Hadash, Einat Kermany, Boaz Carmeli, Ofer Lavi, George Kour, and Alon Jacovi.  Estimate and replace: A novel approach to integrating deep neural networks with existing applications.  _arXiv preprint arXiv:1804.09028_ , 2018.
  * Hansen et al. (2020) Christian Hansen, Casper Hansen, Jakob Grue Simonsen, Birger Larsen, Stephen Alstrup, and Christina Lioma.  Factuality checking in news headlines with eye tracking.  In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 2013–2016, 2020\.
  * Hokamp and Liu (2017) Chris Hokamp and Qun Liu.  Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search.  _arXiv:1704.07138 [cs]_ , May 2017.  URL http://arxiv.org/abs/1704.07138.  arXiv: 1704.07138.
  * Honnibal and Montani (2017) Matthew Honnibal and Ines Montani.  spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing.  _To appear_ , 7(1):411–420, 2017.
  * Honovich et al. (2021) Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend.  $q^2$: Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering.  In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021_ , pages 7856–7870. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.emnlp-main.619.  URL https://doi.org/10.18653/v1/2021.emnlp-main.619.
  * Hu et al. (2019) J Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tongfei Chen, Matt Post, and Benjamin Van Durme.  Improved lexically constrained decoding for translation and monolingual rewriting.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 839–850, 2019.
  * Huang et al. (2020) Luyang Huang, Lingfei Wu, and Lu Wang.  Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5094–5107, 2020.
  * Huang et al. (2016) Ting-Hao Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, et al.  Visual storytelling.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1233–1239, 2016.
  * Jain and Kasbe (2018) Akshay Jain and Amey Kasbe.  Fake news detection.  In _2018 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS)_ , pages 1–5. IEEE, 2018.
  * Jing et al. (2017) Baoyu Jing, Pengtao Xie, and Eric Xing.  On the automatic generation of medical imaging reports.  _arXiv preprint arXiv:1711.08195_ , 2017.
  * Junczys-Dowmunt (2018) Marcin Junczys-Dowmunt.  Dual conditional cross-entropy filtering of noisy parallel corpora.  _arXiv preprint arXiv:1809.00197_ , 2018.
  * Kahn Jr et al. (2009) Charles E Kahn Jr, Curtis P Langlotz, Elizabeth S Burnside, John A Carrino, David S Channin, David M Hovsepian, and Daniel L Rubin.  Toward best practices in radiology reporting.  _Radiology_ , 252(3):852–856, 2009.
  * Kalchbrenner et al. (2014) Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom.  A convolutional neural network for modelling sentences.  _arXiv preprint arXiv:1404.2188_ , 2014.
  * Kang and Hashimoto (2020) Daniel Kang and Tatsunori Hashimoto.  Improved natural language generation via loss truncation.  _arXiv preprint arXiv:2004.14589_ , 2020.
  * Karpukhin et al. (2019) Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, and Marjan Ghazvininejad.  Training on synthetic noise improves robustness to natural noise in machine translation.  _arXiv preprint arXiv:1902.01509_ , 2019.
  * Ke et al. (2021) Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, and Minlie Huang.  Jointgt: Graph-text joint representation learning for text generation from knowledge graphs.  _arXiv preprint arXiv:2106.10502_ , 2021.
  * Kiela (2017) Douwe Kiela.  Deep embodiment: grounding semantics in perceptual modalities.  Technical report, University of Cambridge, Computer Laboratory, 2017.
  * Kim et al. (2020) Hyunwoo Kim, Byeongchang Kim, and Gunhee Kim.  Will I sound like me? improving persona consistency in dialogues through pragmatic self-consciousness.  In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020_ , pages 904–916. Association for Computational Linguistics, 2020.  doi: 10.18653/v1/2020.emnlp-main.65.  URL https://doi.org/10.18653/v1/2020.emnlp-main.65.
  * Koehn (2009) Philipp Koehn.  _Statistical machine translation_.  Cambridge University Press, 2009.
  * Kong et al. (2019) Xiang Kong, Zhaopeng Tu, Shuming Shi, Eduard Hovy, and Tong Zhang.  Neural machine translation with adequacy-oriented learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 6618–6625, 2019.
  * Koto et al. (2021) Fajri Koto, Timothy Baldwin, and Jey Han Lau.  FFCI: A Framework for Interpretable Automatic Evaluation of Summarization.  _arXiv:2011.13662 [cs]_ , July 2021.  URL http://arxiv.org/abs/2011.13662.  arXiv: 2011.13662.
  * Kryściński et al. (2019) Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher.  Evaluating the Factual Consistency of Abstractive Text Summarization.  _arXiv:1910.12840 [cs]_ , October 2019.  URL http://arxiv.org/abs/1910.12840.  arXiv: 1910.12840.
  * Laban et al. (2021a) Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst.  Summac: Re-visiting nli-based models for inconsistency detection in summarization.  _CoRR_ , abs/2111.09525, 2021a.  URL https://arxiv.org/abs/2111.09525.
  * Laban et al. (2021b) Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst.  SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization.  _arXiv:2111.09525 [cs]_ , November 2021b.  URL http://arxiv.org/abs/2111.09525.  arXiv: 2111.09525.
  * Lebret et al. (2016) Rémi Lebret, David Grangier, and Michael Auli.  Neural Text Generation from Structured Data with Application to the Biography Domain.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1203–1213, Austin, Texas, 2016. Association for Computational Linguistics.  doi: 10.18653/v1/D16-1128.
  * Lee et al. (2018) Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo.  Hallucinations in neural machine translation.  2018\.
  * Lee et al. (2021) Seanie Lee, Dong Bok Lee, and Sung Ju Hwang.  CONTRASTIVE LEARNING WITH ADVERSARIAL PER- TURBATIONS FOR CONDITIONAL TEXT GENERATION.  page 25, 2021.
  * Lewis et al. (2019) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  _arXiv preprint arXiv:1910.13461_ , 2019.
  * Li et al. (2018a) Chenliang Li, Weiran Xu, Si Li, and Sheng Gao.  Guiding generation for abstractive text summarization based on key information guide network.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 55–60, 2018a.
  * Li et al. (2018b) Haoran Li, Junnan Zhu, Jiajun Zhang, and Chengqing Zong.  Ensure the Correctness of the Summary: Incorporate Entailment Knowledge into Abstractive Sentence Summarization.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1430–1441, Santa Fe, New Mexico, USA, August 2018b. Association for Computational Linguistics.  URL https://aclanthology.org/C18-1121.
  * Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Georgios P. Spithourakis, Jianfeng Gao, and William B. Dolan.  A persona-based neural conversation model.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_. The Association for Computer Linguistics, 2016a.  doi: 10.18653/v1/p16-1094.  URL https://doi.org/10.18653/v1/p16-1094.
  * Li et al. (2016b) Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky.  Deep reinforcement learning for dialogue generation.  _arXiv preprint arXiv:1606.01541_ , 2016b.
  * Li et al. (2020a) Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston.  Don’t say that! making inconsistent dialogue unlikely with unlikelihood training.  In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 4715–4728. Association for Computational Linguistics, 2020a.  doi: 10.18653/v1/2020.acl-main.428.  URL https://doi.org/10.18653/v1/2020.acl-main.428.
  * Li et al. (2020b) Tian Li, Ahmad Beirami, Maziar Sanjabi, and Virginia Smith.  Tilted empirical risk minimization.  _arXiv preprint arXiv:2007.01162_ , 2020b.
  * Li et al. (2020c) Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, and Haifeng Wang.  Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning.  _arXiv preprint arXiv:2012.15409_ , 2020c.
  * Li et al. (2018c) Yuan Li, Xiaodan Liang, Zhiting Hu, and Eric P Xing.  Hybrid retrieval-generation reinforced agent for medical image report generation.  _Advances in neural information processing systems_ , 31, 2018c.
  * Li et al. (2017) Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li.  Paraphrase generation with deep reinforcement learning.  _arXiv preprint arXiv:1711.00279_ , 2017.
  * Lin (2004) Chin-Yew Lin.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pages 74–81, 2004.
  * Liu et al. (2019a) Guanxiong Liu, Tzu-Ming Harry Hsu, Matthew McDermott, Willie Boag, Wei-Hung Weng, Peter Szolovits, and Marzyeh Ghassemi.  Clinically accurate chest x-ray report generation.  In _Machine Learning for Healthcare Conference_ , pages 249–269. PMLR, 2019a.
  * Liu et al. (2018a) Hairong Liu, Mingbo Ma, Liang Huang, Hao Xiong, and Zhongjun He.  Robust neural machine translation with joint textual and phonetic embedding.  _arXiv preprint arXiv:1810.06729_ , 2018a.
  * Liu et al. (2018b) Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui.  Table-to-text generation by structure-aware seq2seq learning.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018b.
  * Liu et al. (2019b) Tianyu Liu, Fuli Luo, Pengcheng Yang, Wei Wu, Baobao Chang, and Zhifang Sui.  Towards Comprehensive Description Generation from Factual Attribute-value Tables.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5985–5996, Florence, Italy, 2019b. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1600.
  * Liu et al. (2021a) Tianyu Liu, Xin Zheng, Baobao Chang, and Zhifang Sui.  Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 13415–13423, May 2021a.
  * Liu et al. (2021b) Wei Liu, Huanqin Wu, Wenjing Mu, Zhen Li, Tao Chen, and Dan Nie.  Co2sum: Contrastive learning for factual-consistent abstractive summarization.  _arXiv preprint arXiv:2112.01147_ , 2021b.
  * Liu and Lapata (2019) Yang Liu and Mirella Lapata.  Text summarization with pretrained encoders.  _arXiv preprint arXiv:1908.08345_ , 2019.
  * Liu et al. (2021c) Yang Liu, Yifei Sun, and Vincent Gao.  Improving Factual Consistency of Abstractive Summarization on Customer Feedback.  _arXiv:2106.16188 [cs]_ , June 2021c.  URL http://arxiv.org/abs/2106.16188.  arXiv: 2106.16188.
  * Liu et al. (2019c) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ , 2019c.
  * Liu et al. (2020) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742, 2020.
  * Logan IV et al. (2019) Robert L Logan IV, Nelson F Liu, Matthew E Peters, Matt Gardner, and Sameer Singh.  Barack’s wife hillary: Using knowledge-graphs for fact-aware language modeling.  _arXiv preprint arXiv:1906.07241_ , 2019.
  * Longpre et al. (2021) Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh.  Entity-based knowledge conflicts in question answering.  _arXiv preprint arXiv:2109.05052_ , 2021.
  * Ma et al. (2002) Yu-Fei Ma, Lie Lu, Hong-Jiang Zhang, and Mingjing Li.  A user attention model for video summarization.  In _Proceedings of the tenth ACM international conference on Multimedia_ , pages 533–542, 2002.
  * Mao et al. (2020) Yuning Mao, Xiang Ren, Heng Ji, and Jiawei Han.  Constrained abstractive summarization: Preserving factual consistency with constrained generation.  _arXiv preprint arXiv:2010.12723_ , 2020.
  * Maynez et al. (2020) Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald.  On Faithfulness and Factuality in Abstractive Summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1906–1919, Online, July 2020. Association for Computational Linguistics.  doi: 10.18653/v1/2020.acl-main.173.  URL https://www.aclweb.org/anthology/2020.acl-main.173.
  * Meng et al. (2020) Kevin Meng, Damian Jimenez, Fatma Arslan, Jacob Daniel Devasier, Daniel Obembe, and Chengkai Li.  Gradient-based adversarial training on transformer networks for detecting check-worthy factual claims.  _arXiv preprint arXiv:2002.07725_ , 2020.
  * Mesgar et al. (2021) Mohsen Mesgar, Edwin Simpson, and Iryna Gurevych.  Improving Factual Consistency Between a Response and Persona Facts.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 549–562, Online, April 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.eacl-main.44.  URL https://aclanthology.org/2021.eacl-main.44.
  * Mi et al. (2016) Haitao Mi, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah.  Coverage embedding models for neural machine translation.  _arXiv preprint arXiv:1605.03148_ , 2016.
  * Miao et al. (2019) Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li.  Cgmh: Constrained sentence generation by metropolis-hastings sampling.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 6834–6842, 2019.
  * Mishra et al. (2021) Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Lorraine Li, Pavan Kapanipathi, and Kartik Talamadupula.  Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1322–1336, Online, 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.naacl-main.104.  URL https://aclanthology.org/2021.naacl-main.104.
  * Miura et al. (2021) Yasuhide Miura, Yuhao Zhang, Emily Bao Tsai, Curtis P. Langlotz, and Dan Jurafsky.  Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation.  _arXiv:2010.10042 [cs]_ , April 2021.  URL http://arxiv.org/abs/2010.10042.  arXiv: 2010.10042.
  * Miyato et al. (2016) Takeru Miyato, Andrew M Dai, and Ian Goodfellow.  Adversarial training methods for semi-supervised text classification.  _arXiv preprint arXiv:1605.07725_ , 2016.
  * Mnih et al. (2016) Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu.  Asynchronous methods for deep reinforcement learning.  In Maria-Florina Balcan and Kilian Q. Weinberger, editors, _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016_ , volume 48 of _JMLR Workshop and Conference Proceedings_ , pages 1928–1937. JMLR.org, 2016.  URL http://proceedings.mlr.press/v48/mniha16.html.
  * Müller et al. (2019) Mathias Müller, Annette Rios, and Rico Sennrich.  Domain robustness in neural machine translation.  _arXiv preprint arXiv:1911.03109_ , 2019.
  * Nan et al. (2021a) Feng Nan, Cícero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen R. McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, and Bing Xiang.  Improving factual consistency of abstractive summarization via question answering.  In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , pages 6881–6894. Association for Computational Linguistics, 2021a.  doi: 10.18653/v1/2021.acl-long.536.  URL https://doi.org/10.18653/v1/2021.acl-long.536.
  * Nan et al. (2021b) Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen Mckeown, and Bing Xiang.  Entity-level factual consistency of abstractive text summarization.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2727–2733, 2021b.
  * Nie et al. (2019) Feng Nie, Jin-Ge Yao, Jinpeng Wang, Rong Pan, and Chin-Yew Lin.  A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2673–2679, Florence, Italy, 2019. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1256.
  * Nie et al. (2021) Yixin Nie, Mary Williamson, Mohit Bansal, Douwe Kiela, and Jason Weston.  I like fish, especially dolphins: Addressing contradictions in dialogue modeling.  In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , pages 1699–1713. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.acl-long.134.  URL https://doi.org/10.18653/v1/2021.acl-long.134.
  * Nye et al. (2021) Maxwell Nye, Michael Tessler, Josh Tenenbaum, and Brenden M Lake.  Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * Pagnoni et al. (2021) Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov.  Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4812–4829, Online, June 2021. Association for Computational Linguistics.  URL https://www.aclweb.org/anthology/2021.naacl-main.383.
  * Palmer et al. (2005) Martha Palmer, Daniel Gildea, and Paul Kingsbury.  The proposition bank: An annotated corpus of semantic roles.  _Computational linguistics_ , 31(1):71–106, 2005\.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , pages 311–318, 2002.
  * Post and Vilar (2018) Matt Post and David Vilar.  Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1314–1324, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.  doi: 10.18653/v1/N18-1119.  URL https://aclanthology.org/N18-1119.
  * Puduppully et al. (2019) Ratish Puduppully, Li Dong, and Mirella Lapata.  Data-to-text Generation with Entity Modeling.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2023–2035, Florence, Italy, 2019. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1195.
  * Qin et al. (2019) Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao.  Conversing by reading: Contentful neural conversation with on-demand machine reading.  _arXiv preprint arXiv:1906.02738_ , 2019.
  * Qin et al. (2021) Libo Qin, Tianbao Xie, Shijue Huang, Qiguang Chen, Xiao Xu, and Wanxiang Che.  Don’t be contradicted with anything! ci-tod: Towards benchmarking consistency for task-oriented dialogue system.  In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021_ , pages 2357–2367. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.emnlp-main.182.  URL https://doi.org/10.18653/v1/2021.emnlp-main.182.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9, 2019.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_ , 2019.
  * Ranzato et al. (2015) Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba.  Sequence level training with recurrent neural networks.  _arXiv preprint arXiv:1511.06732_ , 2015.
  * Rashkin et al. (2021) Hannah Rashkin, David Reitter, Gaurav Singh Tomar, and Dipanjan Das.  Increasing faithfulness in knowledge-grounded dialogue with controllable features.  In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021_ , pages 704–718. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.acl-long.58.  URL https://doi.org/10.18653/v1/2021.acl-long.58.
  * Raunak et al. (2021) Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt.  The curious case of hallucinations in neural machine translation.  _arXiv preprint arXiv:2104.06683_ , 2021.
  * Rebuffel et al. (2020) Clement Rebuffel, Laure Soulier, Geoffrey Scoutheeten, and Patrick Gallinari.  PARENTing via Model-Agnostic Reinforcement Learning to Correct Pathological Behaviors in Data-to-Text Generation.  In _Proceedings of the 13th International Conference on Natural Language Generation_ , pages 120–130, Dublin, Ireland, 2020. Association for Computational Linguistics.
  * Rebuffel et al. (2022) Clement Rebuffel, Marco Roberti, Laure Soulier, Geoffrey Scoutheeten, Rossella Cancelliere, and Patrick Gallinari.  Controlling hallucinations at word level in data-to-text generation.  _Data Mining and Knowledge Discovery_ , 36(1):318–354, 2022.  ISSN 1573-756X.  doi: 10.1007/s10618-021-00801-4.
  * Reis et al. (2019) Julio CS Reis, André Correia, Fabrício Murai, Adriano Veloso, and Fabrício Benevenuto.  Supervised learning for fake news detection.  _IEEE Intelligent Systems_ , 34(2):76–81, 2019\.
  * Rohrbach et al. (2018) Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko.  Object Hallucination in Image Captioning.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4035–4045, Brussels, Belgium, October 2018. Association for Computational Linguistics.  doi: 10.18653/v1/D18-1437.  URL https://aclanthology.org/D18-1437.
  * Roller et al. (2021) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston.  Recipes for building an open-domain chatbot.  In Paola Merlo, Jörg Tiedemann, and Reut Tsarfaty, editors, _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 - 23, 2021_ , pages 300–325. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.eacl-main.24.  URL https://doi.org/10.18653/v1/2021.eacl-main.24.
  * Sabet et al. (2020) Masoud Jalili Sabet, Philipp Dufter, François Yvon, and Hinrich Schütze.  Simalign: High quality word alignments without parallel training data using static and contextualized embeddings.  _arXiv preprint arXiv:2004.08728_ , 2020.
  * Saito et al. (2020) Itsumi Saito, Kyosuke Nishida, Kosuke Nishida, and Junji Tomita.  Abstractive summarization with combination of pre-trained sequence-to-sequence and saliency models.  _arXiv preprint arXiv:2003.13028_ , 2020.
  * Scialom et al. (2021) Thomas Scialom, Paul-Alexis Dray, Patrick Gallinari, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, and Alex Wang.  QuestEval: Summarization Asks for Fact-based Evaluation.  _arXiv:2103.12693 [cs]_ , April 2021.  URL http://arxiv.org/abs/2103.12693.  arXiv: 2103.12693.
  * Shen et al. (2020) Xiaoyu Shen, Ernie Chang, Hui Su, Cheng Niu, and Dietrich Klakow.  Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7155–7165, Online, 2020. Association for Computational Linguistics.  doi: 10.18653/v1/2020.acl-main.641.
  * Shu et al. (2017) Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu.  Fake news detection on social media: A data mining perspective.  _ACM SIGKDD explorations newsletter_ , 19(1):22–36, 2017.
  * Shu et al. (2019) Kai Shu, Suhang Wang, and Huan Liu.  Beyond news contents: The role of social context for fake news detection.  In _Proceedings of the twelfth ACM international conference on web search and data mining_ , pages 312–320, 2019.
  * Shuster et al. (2021) Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston.  Retrieval augmentation reduces hallucination in conversation.  In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, _Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021_ , pages 3784–3803. Association for Computational Linguistics, 2021.  doi: 10.18653/v1/2021.findings-emnlp.320.  URL https://doi.org/10.18653/v1/2021.findings-emnlp.320.
  * Song et al. (2020a) Haoyu Song, Yan Wang, Wei-Nan Zhang, Zhengyu Zhao, Ting Liu, and Xiaojiang Liu.  Profile Consistency Identification for Open-domain Dialogue Agents.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6651–6662, Online, November 2020a. Association for Computational Linguistics.  doi: 10.18653/v1/2020.emnlp-main.539.  URL https://aclanthology.org/2020.emnlp-main.539.
  * Song et al. (2020b) Haoyu Song, Yan Wang, Weinan Zhang, Xiaojiang Liu, and Ting Liu.  Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation.  In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 5821–5831. Association for Computational Linguistics, 2020b.  doi: 10.18653/v1/2020.acl-main.516.  URL https://doi.org/10.18653/v1/2020.acl-main.516.
  * Song et al. (2020c) Haoyu Song, Wei-Nan Zhang, Jingwen Hu, and Ting Liu.  Generating Persona Consistent Dialogues by Exploiting Natural Language Inference.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):8878–8885, April 2020c.  ISSN 2374-3468, 2159-5399.  doi: 10.1609/aaai.v34i05.6417.  URL http://arxiv.org/abs/1911.05889.  arXiv: 1911.05889.
  * Song et al. (2018) Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea.  A graph-to-sequence model for amr-to-text generation.  _arXiv preprint arXiv:1805.02473_ , 2018.
  * Song et al. (2020d) Yun-Zhu Song, Hong-Han Shuai, Sung-Lin Yeh, Yi-Lun Wu, Lun-Wei Ku, and Wen-Chih Peng.  Attractive or Faithful? Popularity-Reinforced Learning for Inspired Headline Generation.  _arXiv:2002.02095 [cs]_ , February 2020d.  URL http://arxiv.org/abs/2002.02095.  arXiv: 2002.02095.
  * Tang et al. (2019) Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xiaodan Liang, Eric P Xing, and Zhiting Hu.  Target-guided open-domain conversation.  _arXiv preprint arXiv:1905.11553_ , 2019.
  * Tang et al. (2021) Xiangru Tang, Alexander R Fabbri, Ziming Mao, Griffin Adams, Borui Wang, Haoran Li, Yashar Mehdad, and Dragomir Radev.  Investigating crowdsourcing protocols for evaluating the factual consistency of summaries.  _arXiv preprint arXiv:2109.09195_ , 2021.
  * Tian et al. (2019) Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P. Parikh.  Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation.  October 2019.
  * Tu et al. (2016) Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li.  Modeling coverage for neural machine translation.  _arXiv preprint arXiv:1601.04811_ , 2016.
  * Tu et al. (2017) Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li.  Neural machine translation with reconstruction.  In _Thirty-First AAAI Conference on Artificial Intelligence_ , 2017\.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Veličković et al. (2017) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio.  Graph attention networks.  _arXiv preprint arXiv:1710.10903_ , 2017.
  * Vinyals and Le (2015) Oriol Vinyals and Quoc V. Le.  A neural conversational model.  _CoRR_ , abs/1506.05869, 2015.  URL http://arxiv.org/abs/1506.05869.
  * Vinyals et al. (2015) Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan.  Show and tell: A neural image caption generator.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3156–3164, 2015.
  * Wang et al. (2020a) Alex Wang, Kyunghyun Cho, and Mike Lewis.  Asking and answering questions to evaluate the factual consistency of summaries.  In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020_ , pages 5008–5020. Association for Computational Linguistics, 2020a.  doi: 10.18653/v1/2020.acl-main.450.  URL https://doi.org/10.18653/v1/2020.acl-main.450.
  * Wang and Sennrich (2020) Chaojun Wang and Rico Sennrich.  On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3544–3552, Online, July 2020. Association for Computational Linguistics.  doi: 10.18653/v1/2020.acl-main.326.  URL https://aclanthology.org/2020.acl-main.326.
  * Wang (2019) Hongmin Wang.  Revisiting Challenges in Data-to-Text Generation with Fact Grounding.  In _Proceedings of the 12th International Conference on Natural Language Generation_ , pages 311–322, Tokyo, Japan, October 2019\. Association for Computational Linguistics.  doi: 10.18653/v1/W19-8639.
  * Wang et al. (2021) Peng Wang, Junyang Lin, An Yang, Chang Zhou, Yichang Zhang, Jingren Zhou, and Hongxia Yang.  Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4831–4843, Online, 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.findings-acl.427.
  * Wang et al. (2018) Qingyun Wang, Xiaoman Pan, Lifu Huang, Boliang Zhang, Zhiying Jiang, Heng Ji, and Kevin Knight.  Describing a Knowledge Base.  In _Proceedings of the 11th International Conference on Natural Language Generation_ , pages 10–21, Tilburg University, The Netherlands, 2018. Association for Computational Linguistics.  doi: 10.18653/v1/W18-6502.
  * Wang et al. (2020b) Zhenyi Wang, Xiaoyang Wang, Bang An, Dong Yu, and Changyou Chen.  Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1072–1086, Online, 2020b. Association for Computational Linguistics.  doi: 10.18653/v1/2020.acl-main.101.
  * Welleck et al. (2019a) Sean Welleck, Kianté Brantley, Hal Daumé Iii, and Kyunghyun Cho.  Non-monotonic sequential text generation.  In _International Conference on Machine Learning_ , pages 6716–6726. PMLR, 2019a.
  * Welleck et al. (2019b) Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho.  Dialogue natural language inference.  In Anna Korhonen, David R. Traum, and Lluís Màrquez, editors, _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pages 3731–3741. Association for Computational Linguistics, 2019b.  doi: 10.18653/v1/p19-1363.  URL https://doi.org/10.18653/v1/p19-1363.
  * Welleck et al. (2019c) Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho.  Dialogue Natural Language Inference.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3731–3741, Florence, Italy, July 2019c. Association for Computational Linguistics.  doi: 10.18653/v1/P19-1363.  URL https://aclanthology.org/P19-1363.
  * Weng et al. (2017) Rongxiang Weng, Shujian Huang, Zaixiang Zheng, Xinyu Dai, and Jiajun Chen.  Neural machine translation with word predictions.  _arXiv preprint arXiv:1708.01771_ , 2017.
  * Weng et al. (2020a) Rongxiang Weng, Haoran Wei, Shujian Huang, Heng Yu, Lidong Bing, Weihua Luo, and Jiajun Chen.  Gret: Global representation enhanced transformer.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 9258–9265, 2020a.
  * Weng et al. (2020b) Rongxiang Weng, Heng Yu, Xiangpeng Wei, and Weihua Luo.  Towards Enhancing Faithfulness for Neural Machine Translation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2675–2684, Online, November 2020b. Association for Computational Linguistics.  doi: 10.18653/v1/2020.emnlp-main.212.  URL https://www.aclweb.org/anthology/2020.emnlp-main.212.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel R. Bowman.  A broad-coverage challenge corpus for sentence understanding through inference.  In Marilyn A. Walker, Heng Ji, and Amanda Stent, editors, _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_ , pages 1112–1122. Association for Computational Linguistics, 2018.  doi: 10.18653/v1/n18-1101.  URL https://doi.org/10.18653/v1/n18-1101.
  * Williams and Zipser (1989) Ronald J Williams and David Zipser.  A learning algorithm for continually running fully recurrent neural networks.  _Neural computation_ , 1(2):270–280, 1989.
  * Wiseman et al. (2017) Sam Wiseman, Stuart Shieber, and Alexander Rush.  Challenges in Data-to-Document Generation.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2253–2263, Copenhagen, Denmark, 2017. Association for Computational Linguistics.  doi: 10.18653/v1/D17-1239.
  * Wu et al. (2021a) Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Ziqiang Cao, Sujian Li, Hua Wu, and Haifeng Wang.  Bass: Boosting abstractive summarization with unified semantic graph.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6052–6067, 2021a.
  * Wu et al. (2021b) Zeqiu Wu, Michel Galley, Chris Brockett, Yizhe Zhang, Xiang Gao, Chris Quirk, Rik Koncel-Kedziorski, Jianfeng Gao, Hannaneh Hajishirzi, Mari Ostendorf, and Bill Dolan.  A controllable model of grounded response generation.  In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_ , pages 14085–14093. AAAI Press, 2021b.  URL https://ojs.aaai.org/index.php/AAAI/article/view/17658.
  * Xie et al. (2021) Yuexiang Xie, Fei Sun, Yang Deng, Yaliang Li, and Bolin Ding.  Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation.  _arXiv:2108.13134 [cs]_ , September 2021.  URL http://arxiv.org/abs/2108.13134.  arXiv: 2108.13134.
  * Yang et al. (2019) Zonghan Yang, Yong Cheng, Yang Liu, and Maosong Sun.  Reducing word omission errors in neural machine translation: A contrastive learning approach.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6191–6196, 2019.
  * Yates et al. (2007) Alexander Yates, Michele Banko, Matthew Broadhead, Michael J Cafarella, Oren Etzioni, and Stephen Soderland.  Textrunner: open information extraction on the web.  In _Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT)_ , pages 25–26, 2007.
  * Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu.  Bartscore: Evaluating generated text as text generation.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * Zaremba et al. (2014) Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals.  Recurrent neural network regularization.  _arXiv preprint arXiv:1409.2329_ , 2014.
  * Zellers et al. (2020) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi.  Defending Against Neural Fake News.  _arXiv:1905.12616 [cs]_ , December 2020.  URL http://arxiv.org/abs/1905.12616.  arXiv: 1905.12616.
  * Zhang et al. (2020a) Boliang Zhang, Ajay Nagesh, and Kevin Knight.  Parallel corpus filtering via pre-trained language models.  _arXiv preprint arXiv:2005.06166_ , 2020a.
  * Zhang et al. (2021) Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, and Yang Liu.  Neural machine translation with explicit phrase alignment.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 29:1001–1010, 2021.
  * Zhang et al. (2020b) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _International Conference on Machine Learning_ , pages 11328–11339. PMLR, 2020b.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston.  Personalizing dialogue agents: I have a dog, do you have pets too?  In Iryna Gurevych and Yusuke Miyao, editors, _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_ , pages 2204–2213. Association for Computational Linguistics, 2018.  doi: 10.18653/v1/P18-1205.  URL https://aclanthology.org/P18-1205/.
  * Zhang and Ghorbani (2020) Xichen Zhang and Ali A Ghorbani.  An overview of online fake news: Characterization, detection, and discussion.  _Information Processing & Management_, 57(2):102025, 2020.
  * Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan.  Dialogpt: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_ , 2019.
  * Zhang et al. (2020c) Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, and Bill Dolan.  POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training.  _arXiv:2005.00558 [cs]_ , September 2020c.  URL http://arxiv.org/abs/2005.00558.  arXiv: 2005.00558.
  * Zhang et al. (2020d) Yuhao Zhang, Derek Merck, Emily Bao Tsai, Christopher D. Manning, and Curtis P. Langlotz.  Optimizing the Factual Correctness of a Summary: A Study of Summarizing Radiology Reports.  _arXiv:1911.02541 [cs]_ , April 2020d.  URL http://arxiv.org/abs/1911.02541.  arXiv: 1911.02541.
  * Zhao et al. (2020a) Zheng Zhao, Shay B Cohen, and Bonnie Webber.  Reducing quantity hallucinations in abstractive summarization.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2237–2249, 2020a.
  * Zhao et al. (2020b) Zheng Zhao, Shay B. Cohen, and Bonnie Webber.  Reducing Quantity Hallucinations in Abstractive Summarization.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2237–2249, Online, 2020b. Association for Computational Linguistics.  doi: 10.18653/v1/2020.findings-emnlp.203.  URL https://aclanthology.org/2020.findings-emnlp.203.
  * Zheng et al. (2019) Zaixiang Zheng, Shujian Huang, Zhaopeng Tu, Xin-Yu Dai, and Jiajun Chen.  Dynamic past and future for neural machine translation.  _arXiv preprint arXiv:1904.09646_ , 2019.
  * Zhong et al. (2020) Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin.  Neural deepfake detection with factual structure of text.  _arXiv preprint arXiv:2010.07475_ , 2020.
  * Zhou et al. (2021) Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Francisco Guzmán, Luke Zettlemoyer, and Marjan Ghazvininejad.  Detecting Hallucinated Content in Conditional Neural Sequence Generation.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 1393–1404, Online, August 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.findings-acl.120.  URL https://aclanthology.org/2021.findings-acl.120.