* Kumar et al. (2020) Kumar, V., Bhotia, T. S., and Chakraborty, T.  Nurse is closer to woman than surgeon? mitigating gender-biased proximities in word embeddings.  _Trans. Assoc. Comput. Linguistics_ , 8:486–503, 2020.
  * Kuppens et al. (2010) Kuppens, P., Allen, N. B., and Sheeber, L. B.  Emotional inertia and psychological maladjustment.  _Psychological Science_ , 21(7):984–991, 2010\.
  * Kurita et al. (2019) Kurita, K., Vyas, N., Pareek, A., Black, A. W., and Tsvetkov, Y.  Measuring bias in contextualized word representations.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pp. 166–172, Florence, Italy, August 2019. Association for Computational Linguistics.
  * Labutov & Lipson (2013) Labutov, I. and Lipson, H.  Re-embedding words.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , volume 2, pp. 489–493, 2013.
  * Lakkaraju et al. (2014) Lakkaraju, H., Socher, R., and Manning, C.  Aspect specific sentiment analysis using hierarchical deep learning.  In _NIPS Workshop on deep learning and representation learning_ , 2014\.
  * Lal et al. (2019) Lal, Y. K., Kumar, V., Dhar, M., Shrivastava, M., and Koehn, P.  De-mixing sentiment from code-mixed text.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop_ , pp. 371–377, 2019\.
  * Lazarus (1982) Lazarus, R. S.  Thoughts on the relations between emotion and cognition.  _American psychologist_ , 37(9):1019–1024, 1982\.
  * Le & Mikolov (2014) Le, Q. and Mikolov, T.  Distributed representations of sentences and documents.  In _International conference on machine learning_ , pp. 1188–1196, 2014.
  * Li & Lu (2017) Li, H. and Lu, W.  Learning latent sentiment scopes for entity-level sentiment analysis.  In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA_ , pp. 3482–3489. AAAI Press, 2017.
  * Li & Hovy (2017) Li, J. and Hovy, E.  Reflections on sentiment/opinion analysis.  In _A Practical Guide to Sentiment Analysis_ , pp. 41–59. Springer, 2017.
  * Li et al. (2018) Li, J., Jia, R., He, H., and Liang, P.  Delete, retrieve, generate: a simple approach to sentiment and style transfer.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , volume 1, pp. 1865–1874, 2018.
  * Li et al. (2019) Li, X., Bing, L., Li, P., and Lam, W.  A unified model for opinion target extraction and target sentiment prediction.  In _The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019_ , pp. 6714–6721. AAAI Press, 2019\.
  * Liu (2010) Liu, B.  Sentiment analysis and subjectivity.  In Indurkhya, N. and Damerau, F. J. (eds.), _Handbook of Natural Language Processing, Second Edition_ , pp. 627–666. Chapman and Hall/CRC, 2010\.
  * Liu (2012) Liu, B.  _Sentiment Analysis and Opinion Mining_.  Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2012.
  * Liu (2015) Liu, B.  _Sentiment Analysis - Mining Opinions, Sentiments, and Emotions_.  Cambridge University Press, 2015.  ISBN 978-1-10-701789-4.
  * Liu & Zhang (2012) Liu, B. and Zhang, L.  A survey of opinion mining and sentiment analysis.  In _Mining Text Data_ , pp. 415–463. Springer, 2012.
  * Liu et al. (2018) Liu, L., Zhang, D., and Song, W.  Modeling sentiment association in discourse for humor recognition.  In Gurevych, I. and Miyao, Y. (eds.), _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers_ , pp. 586–591. Association for Computational Linguistics, 2018.
  * Liu et al. (2015) Liu, Q., Gao, Z., Liu, B., and Zhang, Y.  Automated rule selection for aspect extraction in opinion mining.  In _Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015_ , pp. 1291–1297. AAAI Press, 2015.
  * Liu et al. (2019) Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V.  Roberta: A robustly optimized BERT pretraining approach.  _CoRR_ , abs/1907.11692, 2019.
  * Lloret et al. (2009) Lloret, E., Balahur, A., Palomar, M., and Montoyo, A.  Towards building a competitive opinion summarization system: Challenges and keys.  In _Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, May 31 - June 5, 2009, Boulder, Colorado, USA, Student Research Workshop and Doctoral Consortium_ , pp. 72–77. The Association for Computational Linguistics, 2009.
  * Lubis et al. (2018) Lubis, N., Sakti, S., Yoshino, K., and Nakamura, S.  Eliciting positive emotion through affect-sensitive dialogue response generation: A neural network approach.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018\.
  * Luo et al. (2019) Luo, H., Li, T., Liu, B., and Zhang, J.  DOER: dual cross-shared RNN for aspect term-polarity co-extraction.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pp. 591–601. Association for Computational Linguistics, 2019.
  * Ma et al. (2018) Ma, Y., Peng, H., and Cambria, E.  Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM.  In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_ , pp. 5876–5883. AAAI Press, 2018.
  * Ma et al. (2020) Ma, Y., Nguyen, K. L., Xing, F. Z., and Cambria, E.  A survey on empathetic dialogue systems.  _Inf. Fusion_ , 64:50–70, 2020.
  * Maas et al. (2011) Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., and Potts, C.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies-volume 1_ , pp. 142–150. Association for Computational Linguistics, 2011.
  * Mairesse & Walker (2007) Mairesse, F. and Walker, M.  Personage: Personality generation for dialogue.  In _Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics_ , pp. 496–503, 2007.
  * Majumder et al. (2019a) Majumder, N., Poria, S., Hazarika, D., Mihalcea, R., Gelbukh, A. F., and Cambria, E.  Dialoguernn: An attentive RNN for emotion detection in conversations.  In _The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019_ , pp. 6818–6825. AAAI Press, 2019a.
  * Majumder et al. (2019b) Majumder, N., Poria, S., Peng, H., Chhaya, N., Cambria, E., and Gelbukh, A. F.  Sentiment and sarcasm classification with multitask learning.  _IEEE Intell. Syst._ , 34(3):38–43, 2019b.
  * Majumder et al. (2020) Majumder, N., Bhardwaj, R., Poria, S., Zadeh, A., Gelbukh, A. F., Hussain, A., and Morency, L.  Improving aspect-level sentiment analysis with aspect extraction.  _CoRR_ , abs/2005.06607, 2020.
  * Martineau & Finin (2009) Martineau, J. and Finin, T.  Delta TFIDF: an improved feature space for sentiment analysis.  In _Proceedings of the Third International Conference on Weblogs and Social Media, ICWSM 2009, San Jose, California, USA, May 17-20, 2009_. The AAAI Press, 2009.
  * Maynard & Greenwood (2014) Maynard, D. and Greenwood, M. A.  Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis.  In _LREC 2014 Proceedings_. ELRA, 2014.
  * McCann et al. (2017) McCann, B., Bradbury, J., Xiong, C., and Socher, R.  Learned in translation: Contextualized word vectors.  In _Advances in Neural Information Processing Systems_ , pp. 6294–6305, 2017.
  * Mei et al. (2007) Mei, Q., Ling, X., Wondra, M., Su, H., and Zhai, C.  Topic sentiment mixture: modeling facets and opinions in weblogs.  In _Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8-12, 2007_ , pp. 171–180. ACM, 2007.
  * Mejova & Srinivasan (2011) Mejova, Y. and Srinivasan, P.  Exploring feature definition and selection for sentiment classifiers.  In _Proceedings of the Fifth International Conference on Weblogs and Social Media, Barcelona, Catalonia, Spain, July 17-21, 2011_. The AAAI Press, 2011.
  * Mikolov et al. (2010) Mikolov, T., Karafiát, M., Burget, L., Černockỳ, J., and Khudanpur, S.  Recurrent neural network based language model.  In _Eleventh annual conference of the international speech communication association_ , 2010.
  * Miller (1995) Miller, G. A.  Wordnet: A lexical database for english.  _Commun. ACM_ , 38(11):39–41, 1995.
  * Mishra et al. (2019a) Mishra, A., Laha, A., Sankaranarayanan, K., Jain, P., and Krishnan, S.  Storytelling from structured data and knowledge graphs : An NLG perspective.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics: Tutorial Abstracts, ACL 2019, Florence, Italy, July 28, 2019, Volume 4: Tutorial Abstracts_ , pp. 43–48. Association for Computational Linguistics, 2019a.
  * Mishra et al. (2019b) Mishra, A., Tater, T., and Sankaranarayanan, K.  A modular architecture for unsupervised sarcasm generation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_ , pp. 6143–6153. Association for Computational Linguistics, 2019b.
  * Mitchell et al. (2013) Mitchell, L., Harris, K. D., Frank, M. R., Dodds, P. S., and Danforth, C. M.  The geography of happiness: Connecting twitter sentiment and expression, demographics, and objective characteristics of place.  _CoRR_ , abs/1302.3299, 2013.
  * Miyato et al. (2017) Miyato, T., Dai, A. M., and Goodfellow, I. J.  Adversarial training methods for semi-supervised text classification.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net, 2017.
  * Moghaddam & Ester (2010) Moghaddam, S. and Ester, M.  Opinion digger: an unsupervised opinion miner from unstructured product reviews.  In _Proceedings of the 19th ACM Conference on Information and Knowledge Management, CIKM 2010, Toronto, Ontario, Canada, October 26-30, 2010_ , pp. 1825–1828. ACM, 2010.
  * Mohammad & Turney (2010) Mohammad, S. and Turney, P.  Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon.  In _Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text_ , pp. 26–34, Los Angeles, CA, June 2010. Association for Computational Linguistics.
  * Mohammad & Turney (2013) Mohammad, S. and Turney, P. D.  Crowdsourcing a word-emotion association lexicon.  _Comput. Intell._ , 29(3):436–465, 2013.
  * Mohammad (2017) Mohammad, S. M.  Challenges in sentiment analysis.  In _A practical guide to sentiment analysis_ , pp. 61–83. Springer, 2017.
  * Moilanen & Pulman (2007) Moilanen, K. and Pulman, S.  Sentiment composition.  In _Proceedings of RANLP_ , volume 7, pp. 378–382, 2007.
  * Moraes et al. (2013a) Moraes, R., Valiati, J. F., and Neto, W. P. G.  Document-level sentiment classification: An empirical comparison between SVM and ANN.  _Expert Syst. Appl._ , 40(2):621–633, 2013a.
  * Moraes et al. (2013b) Moraes, R., Valiati, J. F., and Neto, W. P. G.  Document-level sentiment classification: An empirical comparison between svm and ann.  _Expert Systems with Applications_ , 40(2):621–633, 2013b.
  * Morinaga et al. (2002) Morinaga, S., Yamanishi, K., Tateishi, K., and Fukushima, T.  Mining product reputations on the web.  In _Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining_ , pp. 341–349. ACM, 2002\.
  * Morris & Keltner (2000) Morris, M. W. and Keltner, D.  How emotions work: The social functions of emotional expression in negotiations.  _Research in organizational behavior_ , 22:1–50, 2000.
  * Munikar et al. (2019) Munikar, M., Shakya, S., and Shrestha, A.  Fine-grained sentiment classification using BERT.  _CoRR_ , abs/1910.03474, 2019.
  * Nakagawa et al. (2010) Nakagawa, T., Inui, K., and Kurohashi, S.  Dependency tree-based sentiment classification using crfs with hidden variables.  In _Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics_ , pp. 786–794. Association for Computational Linguistics, 2010.
  * Nakov et al. (2013) Nakov, P., Rosenthal, S., Kozareva, Z., Stoyanov, V., Ritter, A., and Wilson, T.  Semeval-2013 task 2: Sentiment analysis in twitter.  In Diab, M. T., Baldwin, T., and Baroni, M. (eds.), _Proceedings of the 7th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2013, Atlanta, Georgia, USA, June 14-15, 2013_ , pp. 312–320. The Association for Computer Linguistics, 2013.
  * Nasukawa & Yi (2003) Nasukawa, T. and Yi, J.  Sentiment analysis: Capturing favorability using natural language processing.  In _Proceedings of the 2nd international conference on Knowledge capture_ , pp. 70–77. ACM, 2003.
  * Navarretta et al. (2016) Navarretta, C., Choukri, K., Declerck, T., Goggi, S., Grobelnik, M., and Maegaard, B.  Mirroring facial expressions and emotions in dyadic conversations.  In _LREC_ , 2016.
  * Oprea & Magdy (2019) Oprea, S. and Magdy, W.  isarcasm: A dataset of intended sarcasm.  _CoRR_ , abs/1911.03123, 2019.
  * Pamungkas (2019) Pamungkas, E. W.  Emotionally-aware chatbots: A survey.  _CoRR_ , abs/1906.09774, 2019.
  * Pan et al. (2010) Pan, S. J., Ni, X., Sun, J., Yang, Q., and Chen, Z.  Cross-domain sentiment classification via spectral feature alignment.  In _Proceedings of the 19th International Conference on World Wide Web, WWW 2010, Raleigh, North Carolina, USA, April 26-30, 2010_ , pp. 751–760, 2010.
  * Pang & Lee (2004) Pang, B. and Lee, L.  A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts.  In _Proceedings of the 42nd annual meeting on Association for Computational Linguistics_ , pp. 271. Association for Computational Linguistics, 2004.
  * Pang et al. (2002) Pang, B., Lee, L., and Vaithyanathan, S.  Thumbs up?: sentiment classification using machine learning techniques.  In _Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10_ , pp. 79–86. Association for Computational Linguistics, 2002.
  * Partala & Surakka (2004) Partala, T. and Surakka, V.  The effects of affective interventions in human–computer interaction.  _Interacting with computers_ , 16(2):295–309, 2004.
  * Patro et al. (2019) Patro, J., Bansal, S., and Mukherjee, A.  A deep-learning framework to detect sarcasm targets.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_ , pp. 6335–6341. Association for Computational Linguistics, 2019\.
  * Peled & Reichart (2017) Peled, L. and Reichart, R.  Sarcasm SIGN: interpreting sarcasm with sentiment based monolingual machine translation.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers_ , pp. 1690–1700. Association for Computational Linguistics, 2017.
  * Peters et al. (2018) Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_ , pp. 2227–2237. Association for Computational Linguistics, 2018.
  * Polanyi & Zaenen (2006) Polanyi, L. and Zaenen, A.  Contextual valence shifters.  In _Computing attitude and affect in text: Theory and applications_ , pp. 1–10. Springer, 2006.
  * Pontiki et al. (2014) Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., and Manandhar, S.  Semeval-2014 task 4: Aspect based sentiment analysis.  In Nakov, P. and Zesch, T. (eds.), _Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@COLING 2014, Dublin, Ireland, August 23-24, 2014_ , pp. 27–35. The Association for Computer Linguistics, 2014.
  * Pontiki et al. (2016) Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Mohammad, A.-S., Al-Ayyoub, M., Zhao, Y., Qin, B., De Clercq, O., et al.  Semeval-2016 task 5: Aspect based sentiment analysis.  In _Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016)_ , pp. 19–30, 2016.
  * Poria et al. (2016) Poria, S., Cambria, E., and Gelbukh, A. F.  Aspect extraction for opinion mining with a deep convolutional neural network.  _Knowl. Based Syst._ , 108:42–49, 2016.
  * Poria et al. (2017) Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., and Morency, L.-P.  Context-dependent sentiment analysis in user-generated videos.  In _ACL 2017_ , volume 1, pp. 873–883, 2017.
  * Poria et al. (2019) Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., and Mihalcea, R.  MELD: A multimodal multi-party dataset for emotion recognition in conversations.  In Korhonen, A., Traum, D. R., and Màrquez, L. (eds.), _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , pp. 527–536. Association for Computational Linguistics, 2019\.
  * Prendinger & Ishizuka (2005) Prendinger, H. and Ishizuka, M.  The empathic companion: A character-based interface that addresses users’affective states.  _Applied Artificial Intelligence_ , 19(3-4):267–285, 2005.
  * Qiu et al. (2011) Qiu, G., Liu, B., Bu, J., and Chen, C.  Opinion word expansion and target extraction through double propagation.  _Computational Linguistics_ , 37(1):9–27, 2011\.
  * Quan et al. (2019) Quan, W., Zhang, J., and Hu, X. T.  End-to-end joint opinion role labeling with BERT.  In _2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, December 9-12, 2019_ , pp. 2438–2446. IEEE, 2019.
  * Radford et al. (2017) Radford, A., Jozefowicz, R., and Sutskever, I.  Learning to generate reviews and discovering sentiment.  _arXiv preprint arXiv:1704.01444_ , 2017.
  * Raffel et al. (2019) Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _CoRR_ , abs/1910.10683, 2019.
  * Rentoumi et al. (2010) Rentoumi, V., Petrakis, S., Klenner, M., Vouros, G. A., and Karkaletsis, V.  United we stand: Improving sentiment analysis by joining machine learning and rule based methods.  In _LREC_ , 2010.
  * Riloff et al. (2013) Riloff, E., Qadir, A., Surve, P., Silva, L. D., Gilbert, N., and Huang, R.  Sarcasm as contrast between a positive sentiment and negative situation.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL_ , pp. 704–714. ACL, 2013.
  * Rogers et al. (2020) Rogers, A., Kovaleva, O., and Rumshisky, A.  A primer in bertology: What we know about how BERT works.  _CoRR_ , abs/2002.12327, 2020.
  * Rosenthal et al. (2014) Rosenthal, S., Ritter, A., Nakov, P., and Stoyanov, V.  Semeval-2014 task 9: Sentiment analysis in twitter.  In Nakov, P. and Zesch, T. (eds.), _Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@COLING 2014, Dublin, Ireland, August 23-24, 2014_ , pp. 73–80. The Association for Computer Linguistics, 2014.
  * Rosenthal et al. (2015) Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S., Ritter, A., and Stoyanov, V.  Semeval-2015 task 10: Sentiment analysis in twitter.  In Cer, D. M., Jurgens, D., Nakov, P., and Zesch, T. (eds.), _Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2015, Denver, Colorado, USA, June 4-5, 2015_ , pp. 451–463. The Association for Computer Linguistics, 2015.
  * Saadany & Orasan (2020) Saadany, H. and Orasan, C.  Is it great or terrible? preserving sentiment in neural machine translation of arabic reviews.  _arXiv preprint arXiv:2010.13814_ , 2020.
  * Sahlgren & Olsson (2019) Sahlgren, M. and Olsson, F.  Gender bias in pretrained Swedish embeddings.  In _Proceedings of the 22nd Nordic Conference on Computational Linguistics_ , pp. 35–43, Turku, Finland, September–October 2019. Linköping University Electronic Press.
  * Sarma et al. (2018) Sarma, P. K., Liang, Y., and Sethares, B.  Domain adapted word embeddings for improved sentiment classification.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers_ , pp. 37–42, 2018.
  * Schifanella et al. (2016) Schifanella, R., de Juan, P., Tetreault, J. R., and Cao, L.  Detecting sarcasm in multimodal social platforms.  In _Proceedings of the 2016 ACM Conference on Multimedia Conference, MM 2016, Amsterdam, The Netherlands, October 15-19, 2016_ , pp. 1136–1145. ACM, 2016.
  * Sharma et al. (2018) Sharma, R., Bhattacharyya, P., Dandapat, S., and Bhatt, H. S.  Identifying transferable information across domains for cross-domain sentiment classification.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_ , pp. 968–978, 2018.
  * Shen et al. (2017) Shen, T., Lei, T., Barzilay, R., and Jaakkola, T. S.  Style transfer from non-parallel text by cross-alignment.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA_ , pp. 6830–6841, 2017.
  * Shi et al. (2018) Shi, B., Fu, Z., Bing, L., and Lam, W.  Learning domain-sensitive and sentiment-aware word embeddings.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_ , pp. 2494–2504. Association for Computational Linguistics, 2018.
  * Shu et al. (2017) Shu, L., Xu, H., and Liu, B.  Lifelong learning CRF for supervised aspect extraction.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers_ , pp. 148–154. Association for Computational Linguistics, 2017.
  * Socher et al. (2013) Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 conference on empirical methods in natural language processing_ , pp. 1631–1642, 2013.
  * Stolcke (2002) Stolcke, A.  Srilm-an extensible language modeling toolkit.  In _Seventh international conference on spoken language processing_ , 2002.
  * Sundermeyer et al. (2012) Sundermeyer, M., Schlüter, R., and Ney, H.  Lstm neural networks for language modeling.  In _Thirteenth annual conference of the international speech communication association_ , 2012.
  * Taboada et al. (2011) Taboada, M., Brooke, J., Tofiloski, M., Voll, K. D., and Stede, M.  Lexicon-based methods for sentiment analysis.  _Computational Linguistics_ , 37(2):267–307, 2011\.
  * Tai et al. (2015) Tai, K. S., Socher, R., and Manning, C. D.  Improved semantic representations from tree-structured long short-term memory networks.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers_ , pp. 1556–1566. The Association for Computer Linguistics, 2015.
  * Tai & Kao (2013) Tai, Y. and Kao, H.  Automatic domain-specific sentiment lexicon generation with label propagation.  In Weippl, E. R., Indrawan-Santiago, M., Steinbauer, M., Kotsis, G., and Khalil, I. (eds.), _The 15th International Conference on Information Integration and Web-based Applications & Services, IIWAS ’13, Vienna, Austria, December 2-4, 2013_, pp. 53. ACM, 2013.
  * Tan et al. (2009) Tan, S., Cheng, X., Wang, Y., and Xu, H.  Adapting naive bayes to domain adaptation for sentiment analysis.  In _Advances in Information Retrieval, 31th European Conference on IR Research, ECIR 2009, Toulouse, France, April 6-9, 2009. Proceedings_ , volume 5478 of _Lecture Notes in Computer Science_ , pp. 337–349. Springer, 2009.
  * Tang et al. (2014) Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., and Qin, B.  Learning sentiment-specific word embedding for twitter sentiment classification.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , volume 1, pp. 1555–1565, 2014.
  * Tang et al. (2015) Tang, D., Qin, B., and Liu, T.  Document modeling with gated recurrent neural network for sentiment classification.  In _Proceedings of the 2015 conference on empirical methods in natural language processing_ , pp. 1422–1432, 2015.
  * Tang et al. (2016a) Tang, D., Qin, B., Feng, X., and Liu, T.  Effective lstms for target-dependent sentiment classification.  In _COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan_ , pp. 3298–3307. ACL, 2016a.
  * Tang et al. (2016b) Tang, D., Qin, B., and Liu, T.  Aspect level sentiment classification with deep memory network.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_ , pp. 214–224. The Association for Computational Linguistics, 2016b.
  * Tay et al. (2017) Tay, Y., Tuan, L. A., and Hui, S. C.  Dyadic memory networks for aspect-based sentiment analysis.  In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM 2017, Singapore, November 06 - 10, 2017_ , pp. 107–116. ACM, 2017.
  * Tay et al. (2018a) Tay, Y., Luu, A. T., Hui, S. C., and Su, J.  Reasoning with sarcasm by reading in-between.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_ , pp. 1010–1020. Association for Computational Linguistics, 2018a.
  * Tay et al. (2018b) Tay, Y., Luu, A. T., Hui, S. C., and Su, J.  Attentive gated lexicon reader with contrastive contextual co-attention for sentiment classification.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018_ , pp. 3443–3453. Association for Computational Linguistics, 2018b.
  * Thongtan & Phienthrakul (2019) Thongtan, T. and Phienthrakul, T.  Sentiment classification using document embeddings trained with cosine similarity.  In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28 - August 2, 2019, Volume 2: Student Research Workshop_ , pp. 407–414. Association for Computational Linguistics, 2019.
  * Toledo-Ronen et al. (2018) Toledo-Ronen, O., Bar-Haim, R., Halfon, A., Jochim, C., Menczel, A., Aharonov, R., and Slonim, N.  Learning sentiment composition from sentiment lexicons.  In Bender, E. M., Derczynski, L., and Isabelle, P. (eds.), _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018_ , pp. 2230–2241. Association for Computational Linguistics, 2018.
  * Tong (2001) Tong, R. M.  An operational system for detecting and tracking opinions in on-line discussion.  In _Working Notes of the ACM SIGIR 2001 Workshop on Operational Text Classification_ , volume 1, 2001.
  * Tsur et al. (2010) Tsur, O., Davidov, D., and Rappoport, A.  ICWSM - A great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews.  In _Proceedings of the Fourth International Conference on Weblogs and Social Media, ICWSM 2010, Washington, DC, USA, May 23-26, 2010_. The AAAI Press, 2010.
  * Turney (2002) Turney, P. D.  Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA_ , pp. 417–424. ACL, 2002.
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA_ , pp. 5998–6008, 2017.
  * Velikovich et al. (2010) Velikovich, L., Blair-Goldensohn, S., Hannan, K., and McDonald, R. T.  The viability of web-derived polarity lexicons.  In _Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 2-4, 2010, Los Angeles, California, USA_ , pp. 777–785. The Association for Computational Linguistics, 2010.
  * Volkova et al. (2013) Volkova, S., Wilson, T., and Yarowsky, D.  Exploring demographic language variations to improve multilingual sentiment analysis in social media.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pp. 1815–1827, 2013.
  * Vosoughi et al. (2015) Vosoughi, S., Zhou, H., and Roy, D.  Enhanced twitter sentiment classification using contextual information.  In Balahur, A., der Goot, E. V., Vossen, P., and Montoyo, A. (eds.), _Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, WASSA@EMNLP 2015, 17 September 2015, Lisbon, Portugal_ , pp. 16–24. The Association for Computer Linguistics, 2015.
  * Wang & Wan (2018) Wang, K. and Wan, X.  Sentigan: Generating sentimental texts via mixture adversarial networks.  In Lang, J. (ed.), _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden_ , pp. 4446–4452. ijcai.org, 2018.
  * Wang & Manning (2012) Wang, S. I. and Manning, C. D.  Baselines and bigrams: Simple, good sentiment and topic classification.  In _The 50th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, July 8-14, 2012, Jeju Island, Korea - Volume 2: Short Papers_ , pp. 90–94. The Association for Computer Linguistics, 2012.
  * Wang et al. (2016) Wang, Y., Huang, M., Zhu, X., and Zhao, L.  Attention-based LSTM for aspect-level sentiment classification.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_ , pp. 606–615. The Association for Computational Linguistics, 2016.
  * Wiebe (2000) Wiebe, J.  Learning subjective adjectives from corpora.  In Kautz, H. A. and Porter, B. W. (eds.), _Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on on Innovative Applications of Artificial Intelligence, July 30 \- August 3, 2000, Austin, Texas, USA_ , pp. 735–740. AAAI Press / The MIT Press, 2000.
  * Wiebe & Mihalcea (2006) Wiebe, J. and Mihalcea, R.  Word sense and subjectivity.  In _ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006_. The Association for Computer Linguistics, 2006.
  * Wiebe (1994) Wiebe, J. M.  Tracking point of view in narrative.  _Computational Linguistics_ , 20(2):233–287, 1994\.
  * Wiebe et al. (1999) Wiebe, J. M., Bruce, R. F., and O’Hara, T. P.  Development and use of a gold-standard data set for subjectivity classifications.  In _Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics_ , pp. 246–253. Association for Computational Linguistics, 1999.
  * Wiegand & Ruppenhofer (2015) Wiegand, M. and Ruppenhofer, J.  Opinion holder and target extraction based on the induction of verbal categories.  In _Proceedings of the 19th Conference on Computational Natural Language Learning, CoNLL 2015, Beijing, China, July 30-31, 2015_ , pp. 215–225. ACL, 2015.
  * Wilson et al. (2005a) Wilson, T., Hoffmann, P., Somasundaran, S., Kessler, J., Wiebe, J., Choi, Y., Cardie, C., Riloff, E., and Patwardhan, S.  Opinionfinder: A system for subjectivity analysis.  In _Proceedings of hlt/emnlp on interactive demonstrations_ , pp. 34–35. Association for Computational Linguistics, 2005a.
  * Wilson et al. (2005b) Wilson, T., Wiebe, J., and Hoffmann, P.  Recognizing contextual polarity in phrase-level sentiment analysis.  In _HLT/EMNLP 2005, Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, 6-8 October 2005, Vancouver, British Columbia, Canada_ , pp. 347–354. The Association for Computational Linguistics, 2005b.
  * Woodland & Voyer (2011) Woodland, J. and Voyer, D.  Context and intonation in the perception of sarcasm.  _Metaphor and Symbol_ , 26(3):227–239, 2011.
  * Wu & Huang (2016) Wu, F. and Huang, Y.  Sentiment domain adaptation with multiple sources.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_. The Association for Computer Linguistics, 2016\.
  * Wu et al. (2016) Wu, H., Gu, Y., Sun, S., and Gu, X.  Aspect-based opinion summarization with convolutional neural networks.  In _2016 International Joint Conference on Neural Networks, IJCNN 2016, Vancouver, BC, Canada, July 24-29, 2016_ , pp. 3157–3163. IEEE, 2016.
  * Wu et al. (2012) Wu, W., Li, H., Wang, H., and Zhu, K.  Probase: A probabilistic taxonomy for text understanding.  _Proceedings of the ACM SIGMOD International Conference on Management of Data_ , 05 2012.
  * Xiang et al. (2010) Xiang, E. W., Cao, B., Hu, D. H., and Yang, Q.  Bridging domains using world wide knowledge for transfer learning.  _IEEE Trans. Knowl. Data Eng._ , 22(6):770–783, 2010.
  * Xie et al. (2019) Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., and Le, Q. V.  Unsupervised data augmentation.  _arXiv preprint arXiv:1904.12848_ , 2019.
  * Yang & Cardie (2013) Yang, B. and Cardie, C.  Joint inference for fine-grained opinion extraction.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, 4-9 August 2013, Sofia, Bulgaria, Volume 1: Long Papers_ , pp. 1640–1649. The Association for Computer Linguistics, 2013.
  * Yu & Hatzivassiloglou (2003) Yu, H. and Hatzivassiloglou, V.  Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences.  In _Proceedings of the 2003 conference on Empirical methods in natural language processing_ , pp. 129–136. Association for Computational Linguistics, 2003.
  * Zadeh et al. (2016) Zadeh, A., Zellers, R., Pincus, E., and Morency, L.  Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages.  _IEEE Intelligent Systems_ , 31(6):82–88, 2016\.
  * Zadeh et al. (2017) Zadeh, A., Chen, M., Poria, S., Cambria, E., and Morency, L.-P.  Tensor fusion network for multimodal sentiment analysis.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pp. 1103–1114, 2017.
  * Zadeh et al. (2018a) Zadeh, A., Liang, P. P., Mazumder, N., Poria, S., Cambria, E., and Morency, L.-P.  Memory fusion network for multi-view sequential learning.  In _AAAI_ , pp. 5634–5641, 2018a.
  * Zadeh et al. (2018b) Zadeh, A., Liang, P. P., Poria, S., Cambria, E., and Morency, L.  Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_ , pp. 2236–2246. Association for Computational Linguistics, 2018b.
  * Zadeh et al. (2018c) Zadeh, A., Liang, P. P., Poria, S., Vij, P., Cambria, E., and Morency, L.-P.  Multi-attention recurrent network for human communication comprehension.  In _AAAI_ , pp. 5642–5649, 2018c.
  * Zajonc (1980) Zajonc, R. B.  Feeling and thinking: Preferences need no inferences.  _AMERICAN PSYCHOLOGIST_ , pp. 151–175, 1980.
  * Zang & Wan (2017) Zang, H. and Wan, X.  Towards automatic generation of product reviews from aspect-sentiment scores.  In _Proceedings of the 10th International Conference on Natural Language Generation, INLG 2017, Santiago de Compostela, Spain, September 4-7, 2017_ , pp. 168–177. Association for Computational Linguistics, 2017.
  * Zhang et al. (2018a) Zhang, L., Wang, S., and Liu, B.  Deep learning for sentiment analysis: A survey.  _Wiley Interdiscip. Rev. Data Min. Knowl. Discov._ , 8(4), 2018a.
  * Zhang et al. (2019a) Zhang, M., Liang, P., and Fu, G.  Enhancing opinion role labeling with semantic-aware word representations from semantic role labeling.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pp. 641–646. Association for Computational Linguistics, 2019a.
  * Zhang et al. (2019b) Zhang, S., Zhang, X., Chan, J., and Rosso, P.  Irony detection via sentiment-based transfer learning.  _Inf. Process. Manag._ , 56(5):1633–1644, 2019b.
  * Zhang et al. (2015) Zhang, X., Zhao, J. J., and LeCun, Y.  Character-level convolutional networks for text classification.  In _Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada_ , pp. 649–657, 2015.
  * Zhang et al. (2018b) Zhang, Z., Han, J., Coutinho, E., and Schuller, B.  Dynamic difficulty awareness training for continuous emotion prediction.  _IEEE Transactions on Multimedia_ , 21(5):1289–1301, 2018b.
  * Zhao et al. (2018a) Zhao, H., Zhang, S., Wu, G., Moura, J. M. F., Costeira, J. P., and Gordon, G. J.  Adversarial multiple source domain adaptation.  In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada_ , pp. 8568–8579, 2018a.
  * Zhao et al. (2018b) Zhao, J., Wang, T., Yatskar, M., Ordonez, V., and Chang, K.-W.  Gender bias in coreference resolution: Evaluation and debiasing methods.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pp. 15–20, New Orleans, Louisiana, June 2018b. Association for Computational Linguistics.
  * Zhao et al. (2018c) Zhao, J., Zhou, Y., Li, Z., Wang, W., and Chang, K.-W.  Learning gender-neutral word embeddings.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 4847–4853, Brussels, Belgium, October-November 2018c. Association for Computational Linguistics.
  * Zhao et al. (2019) Zhao, J., Wang, T., Yatskar, M., Cotterell, R., Ordonez, V., and Chang, K.-W.  Gender bias in contextualized word embeddings.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 629–634, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * Zhou et al. (2018) Zhou, H., Huang, M., Zhang, T., Zhu, X., and Liu, B.  Emotional chatting machine: Emotional conversation generation with internal and external memory.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018\.
  * Zhu & Ghahramani (2002) Zhu, X. and Ghahramani, Z.  Learning from labeled and unlabeled data with label propagation.  _CMU CALD tech report CMU-CALD-02-107_ , 2002.
  * Ziser & Reichart (2018) Ziser, Y. and Reichart, R.  Pivot based language modeling for improved neural domain adaptation.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_ , pp. 1241–1251. Association for Computational Linguistics, 2018.

|  Soujanya Poria is an assistant professor of Information Systems Technology
and Design, at the Singapore University of Technology and Design (SUTD),
Singapore. He holds a Ph.D. degree in Computer Science from the University of
Stirling, UK. He is a recipient of the prestigious early career research award
called “NTU Presidential Postdoctoral Fellowship” in 2018. Soujanya has co-
authored more than 100 research papers, published in top-tier conferences and
journals such as ACL, EMNLP, AAAI, NAACL, Neurocomputing, Computational
Intelligence Magazine, etc. Soujanya has been an area chair at top conferences
such as ACL, EMNLP, NAACL. Soujanya serves or has served on the editorial
boards of the Cognitive Computation and Information Fusion.
---|---
|  Devamanyu Hazarika is a senior Ph.D. candidate at NUS, Singapore. His
primary research is on analyzing the role of context in multimodal affective
systems. Devamanyu has published more than 25 research papers in top-tier
conferences and journals such as ACL, EMNLP, AAAI, NAACL, Information Fusion,
etc. He is also one of the prestigious presidential Ph.D. scholarship holders
at NUS. He has been awarded the NUS Research Achievement Award and has been a
PC-member in many top-tier conferences and journals.
---|---
|  Navonil Majumder is a research fellow at the Singapore University of
Technology and Design (SUTD). He has received his MSc and Ph.D. degrees from
CIC-IPN in 2017 and 2020, respectively. Navonil was awarded the Lázaro
Cárdenas Prize for his MSc and Ph.D. studies. He has also received the best
Ph.D. thesis award by the Mexican Society for Artificial Intelligence (SMIA).
His research interests encompass natural language processing, dialogue
understanding, sentiment analysis, and multimodal language processing. Navonil
has more than 25 research publications in top-tier conferences and journals,
such as, ACL, EMNLP, AAAI, Knowledge-Based Systems, IEEE Intelligent Systems,
etc. Navonil’s research works have occasionally been featured in news portals
such as Techtimes, Datanami, etc.
---|---
|  Rada Mihalcea is a Professor of Computer Science and Engineering at the
University of Michigan and the Director of the Michigan Artificial
Intelligence Lab. Her research interests are in lexical semantics,
multilingual NLP, and computational social sciences. She serves or has served
on the editorial boards of the Journals of Computational Linguistics, Language
Resources and Evaluations, Natural Language Engineering, Journal of Artificial
Intelligence Research, IEEE Transactions on Affective Computing, and
Transactions of the Association for Computational Linguistics. She was a
program co-chair for EMNLP 2009 and ACL 2011, and a general chair for NAACL
2015 and *SEM 2019. She currently serves as the ACL Vice-President Elect. She
is the recipient of an NSF CAREER award (2008) and a Presidential Early Career
Award for Scientists and Engineers awarded by President Obama (2009). In 2013,
she was made an honorary citizen of her hometown of Cluj-Napoca, Romania.
---|---


# On using distributed representations of source code for the
detection of C security vulnerabilities

David Coimbra, Sofia Reis, Rui Abreu, Corina Păsăreanu, Hakan Erdogmus
INESC-ID & IST/U.Lisbon, Portugal
INESC-ID & FEUP, Portugal
Carnegie Mellon University, USA

###### Abstract

This paper presents an evaluation of the code representation model Code2vec
when trained on the task of detecting security vulnerabilities in C source
code. We leverage the open-source library astminer to extract path-contexts
from the abstract syntax trees of a corpus of labeled C functions. Code2vec is
trained on the resulting path-contexts with the task of classifying a function
as vulnerable or non-vulnerable. Using the CodeXGLUE benchmark, we show that
the accuracy of Code2vec for this task is comparable to simple transformer-
based methods such as pre-trained RoBERTa, and outperforms more naive NLP-
based methods. We achieved an accuracy of $61.43\%$ while maintaining low
computational requirements relative to larger models.

##  1 Introduction

Security vulnerabilities are a major concern in software development, as even
the simplest mistakes can be turned into attack vectors by a malicious party.
In continuous integration (CI), it is common to introduce static analysers
into the build pipeline to verify code against known patterns [?; ?; ?]. For
example, Brakeman111https://brakemanscanner.org/ and
SonarQube222https://docs.sonarqube.org/latest/analysis/github-integration/ are
static analysers capable of detecting software vulnerabilities in source code
that can be used for this purpose.

Static analyzers can reduce security concerns in the software development
lifecycle when introduced in the implementation phase, to assist developers to
produce safer software. These analyzers are also useful tools for code review.
Nowadays, one common practice to provide feedback is using GitHub pull
requests. Warnings are usually added automatically to pull requests as
comments, which can be reviewed manually before merging. Such pipelines
streamline the quality assurance process and increase productivity. However,
current techniques in static analysis are limited: they are prone to false
positives (wasting developer effort), and false negatives (making the analysis
less reliable) [?; ?].

In recent years, there has been an increase in the application of statistical
models, namely neural networks, to a variety of code intelligence tasks,
including vulnerability detection [?; ?]. This research has mainly focused on
the application of pre-trained models that capture knowledge particular to a
specific programming language, and has been inspired by transformer-based
models [?] such as BERT [?] and GPT [?], both developed in the context of
natural language processing (NLP). For example, models such as CodeBERT [?],
C-BERT [?] and PLBART [?] produce distributed representations from source code
that have been applied to many code tasks, such as code search, code
translation and vulnerability detection. The recent CodeXGLUE benchmark [?]
aims to facilitate the comparison and evaluation of these recent models in a
large variety of tasks. This benchmark is publicly available and open for for
further contributions 333https://github.com/microsoft/CodeXGLUE.

Although these models are demonstrably effective, they also have their
limitations: mainly, large models with hundreds of millions of parameters need
a relatively large amount of computational resources, including both memory
and CPU time [?]. The requirement for large amounts of computational resources
is a significant limitation for researchers and developers, and it can make
the usage of large pre-trained models impractical. There is a noticeable
trade-off between model representativeness and convenience of use. As a
result, often smaller models are preferred for detection performance.

⬇

void scsi_req_abort(SCSIRequest *req, int status) {

if (!req->enqueued) {

return;

}

scsi_req_ref(req);

scsi_req_dequeue(req);

req->io_canceled = true;

if (req->ops->cancel_io) {

req->ops->cancel_io(req);

}

scsi_req_complete(req, status);

scsi_req_unref(req);

}

Figure 1: Example of non-vulnerable function.

⬇

uint32_t HELPER(shr_cc)(CPUM68KState *env, uint32_t val, uint32_t shift) {

uint64_t temp;

uint32_t result;

shift &= 63;

temp = (uint64_t)val << 32 >> shift;

result = temp >> 32;

env->cc_c = (temp >> 31) & 1;

env->cc_n = result;

env->cc_z = result;

env->cc_v = 0;

env->cc_x = shift ? env->cc_c : env->cc_x;

return result;

}

Figure 2: Example of vulnerable function.

In this work, we investigated the applicability of a code representation
model, Code2vec [?], to the vulnerability detection task. Code2vec is built on
a simple attention-based feed-forward neural network that learns and combines
semantic knowledge extracted from AST path-contexts. To the best of our
knowledge, Code2vec has not been evaluated for vulnerability detection. We
evaluated Code2vec on a labeled corpus of C functions. We found that Code2vec
outperformed traditional NLP-based approaches for vulnerability detection at
an accuracy of $61.43\%$, comparable to simple transformer models such as pre-
trained RoBERTa [?]. We show that these results are achievable at a fraction
of the computational resources and training time necessary for transformer-
based models; Code2vec ran a full training session in approximately $5$
minutes on a consumer-grade GPU, handling $1024$ samples of data at each
training step without exhausting its memory. We submitted our results to the
CodeXGLUE leaderboard for the defect detection task for comparison with the
state-of-the-art.

Our contributions are as follows:

  * •

An evaluation of Code2vec on the task of vulnerability detection using a
dataset of labeled C functions both regarding accuracy and computational
requirements (training time and memory).

  * •

A replication package with the scripts and data to train and test the model,
for reproducibility. Available online: https://github.com/dcoimbra/dx2021.

The paper is organized as follows: in section 2, we present our approach for
applying Code2vec to vulnerability detection. In section 3, we describe the
implementation details related to our extraction of path-contexts and Code2vec
configuration. In section 3, we describe the evaluation metrics employed and
present our results alongside previous studies, and discuss them in section 5.
In section 6, we give a brief summary of the related work in deep learning for
vulnerability detection. Finally, section 7 presents our conclusions and lays
discusses future work.

##  2 Approach

In this section, we describe our approach for using path-context
representations for the detection of security vulnerabilities using Code2vec.
We describe our procedure for the extraction of path-contexts from a corpus of
labeled functions and provide a summary of Code2vec and how we adapted it for
the vulnerability detection task.

###  2.1 Dataset

| Vulnerable | Non-Vulnerable
---|---|---
Train | $10018$ | $11836$
Test | $1255$ | $1477$
Validation | $1187$ | $1545$
Table 1: Distribution of vulnerable and non-vulnerable functions in Devign.

We used the public dataset Devign444https://sites.google.com/view/devign [?],
provided as part of the CodeXGLUE benchmark. Devign includes $27318$ manually-
labeled functions collected from QEMU and FFmpeg, two large C open-source
projects. These functions were extracted by collecting security-related
commits and selecting vulnerable and non-vulnerable versions of functions from
the labeled commits. Each function was manually labeled by a group of three
professional security experts. Functions are labeled as $1$ (vulnerable) and
$0$ (non-vulnerable), with no distinction made with regard to the type of
vulnerability. Examples of a non-vulnerable and vulnerable functions extracted
from Devign are displayed in Figures 2 and 2 respectively. We used the dataset
splits as prepared by
CodeXGLUE555https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/Defect-
detection: $80\%$/$10\%$/$10\%$ for training, validation, and testing,
respectively. The distribution of vulnerable and non-vulnerable functions for
the dataset is presented in Table 1.

Figure 3: Code2vec architecture. Adapted from original image by [?]

###  2.2 Representing code snippets as bags of path-contexts

The path-attention model on which Code2vec is built takes a source code
function as represented by a set of _path-contexts_ , extracted from its
abstract syntax tree (AST). We describe the definitions of AST, AST path and
path-context as presented in the Code2vec paper:

######  Definition 2.1 (Abstract Syntax Tree).

An abstract syntax tree (AST) for a code snippet $\mathcal{C}$ is a tuple
$<N,T,X,s,\delta,\phi>$ where $N$ is a set of non-terminal nodes, $T$ is a set
of terminal nodes, $X$ is a set of values, $s\in N$ is the root node,
$\delta:N\xrightarrow{}(N\cup T)^{*}$ is a function that maps a non-terminal
node to a list of its children, and $\phi:T\xrightarrow{}X$ is a function that
maps a terminal node to its associated value. Every node except the root
appears exactly once in all the lists of children; that is, each node has
exactly one parent.

######  Definition 2.2 (AST path).

An AST path of length $k$ is a sequence of the form: $n_{1}d_{1}\dots
n_{k}d_{k}n_{k+1},$ where $n_{1},n_{k+1}\in T$ are terminal nodes,
$\forall{i}\in[2..k]:n_{i}\in N$ are non-terminal nodes and
$\forall{i}\in[1..k]:d_{i}\in\\{\uparrow,\downarrow\\}$ are movement
directions (up or down in the tree). If $d_{i}=\uparrow$, then
$n_{i}\in\delta(n_{i+1})$; if $d_{i}=\downarrow$, then
$n_{i+1}\in\delta(n_{i})$.

######  Definition 2.3 (Path-context).

Given an AST path $p$, a path-context is a triplet $<x_{s},p,x_{t}>$ where $p$
is a syntactic path in the AST and $x_{s}$ and $x_{t}$ correspond to the
values associated with the start and end terminals of $p$, respectively. A
possible path-context for the statement x = 7 would be:

$\scriptstyle<\texttt{x},(NameExpr\uparrow AssignExpr\downarrow
IntegerLiteralExpr),\texttt{7}>$

It is possible to limit the paths by a maximum length and a maximum width
(distance between two child nodes of the same intermediate node). A code
snippet $\mathcal{C}$ is represented as a _bag of path-contexts_ consisting of
path-contexts extracted from the AST for $\mathcal{C}$. We kept the Code2vec
defaults of maximum length $8$ and maximum width $3$ and, for each function in
the corpus, extracted a bag of at most $200$ path-contexts.

###  2.3 The Code2vec model

Code2vec learns embedding matrices for paths, values and labels
($\boldsymbol{path\\_vocab}$, $\boldsymbol{value\\_vocab}$,
$\boldsymbol{tags\\_vocab}$, respectively), a fully-connected layer, and an
attention vector $\boldsymbol{a}$. An illustration of the Code2vec
architecture is shown in Figure 3. An embedding for a single path-context
$b_{i}=<x_{s},p_{j},x_{t}>$ is a _context vector_ $\boldsymbol{c_{i}}$ which
corresponds to the concatenation of the embeddings of the start and end tokens
and of their connecting paths:

$\displaystyle\boldsymbol{c_{i}}=\mathrm{embedding}(<x_{s},p_{j},x_{t}>)=$ (1)
$\displaystyle\scriptstyle[\boldsymbol{value\\_vocab}_{s};\boldsymbol{path\\_vocab}_{j};\boldsymbol{value\\_vocab}_{t}]\in\mathbb{R}^{3d}$

In the previous equation, the operator ; is the concatenation operator and $d$
is an empirically-determined hyperparameter defining the length of the
internal representation.

A fully connected layer of Code2vec learns to combine each component of the
embedding of a path-context, through a simple linear combination with a
learned weights matrix $\boldsymbol{W}$ passed through a hyperbolic tangent
function:

$\boldsymbol{\tilde{c_{i}}}=\tanh(\boldsymbol{W}\cdot\boldsymbol{c_{i}})\in\mathbb{R}^{d}$
(2)

In the previous equation, $\boldsymbol{W}\in\mathbb{R}^{3d\times d}$. Finally,
Code2vec’s attention mechanism aggregates all combined context vectors
$\\{\boldsymbol{\tilde{c}_{1}},\dots,\boldsymbol{\tilde{c}_{n}}\\}$ into a
single representation. The attention weight $\alpha_{i}$ of each
$\boldsymbol{\tilde{c}_{i}}$ is obtained through the normalized inner product
between $\boldsymbol{\tilde{c}_{i}}$ and the global attention vector
$\boldsymbol{a}$.

$\alpha_{i}=\frac{\mathrm{exp(\boldsymbol{\tilde{c}_{i}^{\intercal}}\cdot\boldsymbol{a})}}{\sum^{n}_{j=1}\mathrm{exp(\boldsymbol{\tilde{c}_{j}^{\intercal}}\cdot\boldsymbol{a})}}$
(3)

The final code vector $\boldsymbol{v}$ is a weighted average of the combined
context vectors factored by their attention weights:

$\boldsymbol{v}=\sum^{n}_{i=1}\alpha_{i}\cdot\boldsymbol{\tilde{c}_{i}}$ (4)

For prediction, the probability that a specific label $y_{i}$ is assigned to a
code snippet $\mathcal{C}$ is the normalized inner product between the
embedding for $y_{i}$ and the code vector $\boldsymbol{v}$:

$\forall{y_{i}}\in
Y:q(y_{i})=\frac{\mathrm{exp}(\boldsymbol{v^{\intercal}}\cdot\boldsymbol{tags\\_vocab}_{i})}{\sum_{y_{j}\in
Y}\mathrm{exp}(\boldsymbol{v^{\intercal}}\cdot\boldsymbol{tags\\_vocab}_{j})}$
(5)

In the previous equation, $Y$ is the set of label values found in the training
corpus. Training is performed by minimizing cross-entropy loss using the Adam
optimization algorithm. For inference, we take the target label to which
Code2vec assigned the highest probability.

##  3 Implementation

In this section, we describe the implementation details for our approach. We
describe the open-source library astminer, which we leveraged for extracting
bags of path-contexts, as well as the parameters used for extraction and
training. Our final pipeline is illustrated in Figure 4.

Figure 4: Experiments pipeline. The vulnerable and non-vulnerable raw source
code functions are independently passed to astminer. For each function,
astminer extracts the respective AST, from which it computes an appropriate
bag of path-contexts. After labeling each bag of path-contexts corresponding
to each function, the result is passed to Code2vec for training. The trained
model is then used for inference.

###  3.1 Extracting bags of path-contexts

To extract a set of path-contexts for each code snippet in Devign, we use the
open-source library astminer666https://github.com/JetBrains-Research/astminer
[?]. We wrote a custom script that visits each function in the corpus, and
computes its path-contexts. As Code2vec is designed to predict the names of
functions rather than numeric labels, our script converts each label of $0$ or
$1$ in the original dataset to the string tokens “‘safe”’ or “‘vuln”’
respectively. Following Code2vec defaults, we limit the length and width of
each path-context to $8$ and $3$ respectively, and extract at most $200$ path-
contexts per function.

By default, astminer replaces each value and path in a path-context with a
corresponding ID number in order to reduce memory consumption and training
time when passing the samples to Code2vec. This is implemented by maintaining
tables of $<\mathrm{id},\mathrm{value}>$ pairs for tokens, node types, and
paths throughout the entire run. However, this was considerably memory-
intensive on the machine we used. As such, we bypassed this feature and
directly extracted path-contexts in their original format. We computed the MD5
hash of the string representation of the path component of each path-context
instead of extracting the entire path as-is. This process is identical to the
one used in the Code2vec paper for function name prediction [?].

| Vulnerable | Non-Vulnerable
---|---|---
Train | 9987 | 11809
Test | 1253 | 1472
Validation | 1185 | 1541
Table 2: Distribution of vulnerable and non-vulnerable functions in Devign
after applying astminer.

The astminer library was unable to extract path-contexts from $71$ functions
of the Devign dataset. Therefore, these samples were not included in our
study. The final distribution of functions after applying astminer is
described in Table 2: $9987$ vulnerable functions and $11809$ non-vulnerable
functions for the training dataset; $1253$ vulnerable functions and $1473$
non-vulnerable functions for the testing dataset; and, finally, $1185$
vulnerable functions and $1541$ non-vulnerable functions for the validation
dataset.

###  3.2 Code2vec

We used the official implementation of Code2vec777https://github.com/tech-
srl/Code2vec. We trained for $20$ epochs and performed inference on the epoch
with the highest F1-score on the validation dataset. The performance measures
of the model were adapted to the vulnerability detection task: we considered a
prediction of “‘safe”’ to be a negative prediction while a prediction of
“‘vuln”’ to be a positive prediction. The training hyperparameters followed
Code2vec defaults: batch size of $1024$, embedding size of $128$, and dropout
rate of $0.25$.

##  4 Evaluation

Model | Accuracy | Precision | Recall | F1
---|---|---|---|---
CoTexT | $66.62$ | - | - | -
C-BERT | $65.45$ | - | - | -
PLBART | $63.18$ | - | - | -
CodeBERT | $62.08$ | - | - | -
Code2vec | 61.43 | 57.50 | 61.77 | 59.56
RoBERTa | $61.05$ | - | - | -
TextCNN | $60.69$ | - | - | -
BiLSTM | $59.37$ | - | - | -
Table 3: Results for the Devign dataset alongside the CodeXGLUE leaderboard. Our contributions are in bold. Model | Train Time | #Epochs | Hardware
---|---|---|---
Code2vec | 5 minutes | 20 | 1050Ti x1
CodeBERT | 7 hours | 5 | 1050Ti x1
CodeBERT | 1 hour | 5 | Tesla P100 x2
Table 4: Training time on the Devign dataset alongside the CodeXGLUE measurement, for a complete training session. Our contributions are in bold. Model | #Params | Embedding size | Memory (MB)
---|---|---|---
Code2vec | 31M | $128$ | $600$
CodeBERT | 125M | $400$ | $2484$
Table 5: Parameter count and memory consumption for each model. Memory
consumption is defined as the amount occupied in RAM by the model and a single
sample of data during a training step, with all gradients loaded.

The CodeXGLUE benchmark for defect detection reports only accuracy as an
evaluation metric. As Devign is a balanced dataset, accuracy is an appropriate
metric for assessing performance. Nevertheless, we also computed precision,
recall and F1-score in addition to accuracy. These metrics help us assess the
model’s ability to distinguish between vulnerable and non-vulnerable samples.
Our results are shown in Table 3 alongside the current entries in the
CodeXGLUE leaderboard. We note that CoTexT had not been released at the time
of writing. In addition to these scores, we compared the training time and
memory consumption for Code2vec and CodeBERT on this task on our hardware. We
computed training time for a complete training session for each model, which
corresponds to $20$ epochs on Code2vec and $5$ epochs on CodeBERT. We chose
$5$ epochs for CodeBERT as this is the value used in CodeXGLUE to obtain the
original results. Training times for each model are presented in Table 4 while
memory consumption is shown in Table 5.

##  5 Results and Discussion

Based on accuracy alone, Code2vec outperformed the traditional NLP-based
methods BiLSTM [?] ($61.43>59.37$) and TextCNN [?] ($61.43>60.69$). The
obtained accuracy score for Code2vec was slightly higher than pre-trained
RoBERTa ($61.43>61.05$): the two methods have similar performance. This is to
be expected as these models were not designed for code intelligence tasks, nor
pre-trained on source code data. In the same vein, Code2vec was outperformed
by PLBART ($61.43<63.18$), C-BERT ($61.43<65.45$) and CodeBERT
($61.43<62.08$), which uses the transformer architecture to learn source-code
features through pre-training on large amounts of source code data. As an
advantage, transformer-based models do not require an intermediate
representation and can be fine-tuned on source code directly.

Regarding training time, as shown in Table 4, Code2vec completed a $20$-epoch
training session in approximately $5$ minutes on our hardware. Conversely, on
the same hardware, CodeBERT completes a $5$-epoch training session in
approximately $7$ hours. This is to be expected, as transformer-based models
need a larger amount of parameters, in turn requiring much more computational
resources to process the large amounts of data. Additionally, transformer-
based models create internal representations that occupy large amounts of
memory, which typically are not available on consumer-grade hardware:
therefore, training has to be carried out in very small batches of data,
increasing training time. A large advantage of Code2vec compared with
transformer-based models is its relatively low memory footprint: as shown in
Table 5, a single training step with CodeBERT requires approximately $2.5$GB
of memory to complete one training step, an amount mostly represented by saved
gradients during back-propagation. Conversely, a single training step with
Code2vec was performed with just $600$MB of GPU memory. This is due to
Code2vec’s simpler architecture, allowing for a lower amount of gradients to
be saved during training, as well as a lower number of parameters and smaller
embedding sizes. Additionally, Code2vec’s lower memory footprint allows us to
load larger batches of data into memory at each training step, increasing
training performance.

##  6 Related Work

Recent research on machine learning for security vulnerability detection has
used both token-based and graph-based approaches [?; ?; ?].

Token-based models consider the code as a sequence of tokens. Several models
have been proposed using different neural network architectures such as
Bidirectional Long-Short-Term Memory (BSLTM) [?], Convolutional Neural
Networks (CNN) [?], and Recurrent Neural Networks (RNN). However, simple
token-based models struggle to reason about the long sequences produced from
transforming source code into token sequences. To help address this problem,
newer approaches using code slices instead of the entire code sequences were
proposed, see for example VulDeePecker [?] and SySeVR [?]. The hypothesis
behind slicing is that different parts of the code are not equally important
for the model to learn vulnerability patterns. Therefore, these newer
approaches consider only slices extracted from _interesting points_ (e.g., API
calls)—points considered important for vulnerability prediction. The rest of
the code elements are ignored. Token-based approaches usually fail to maintain
the dependencies between the tokens that are the root of the problem. Thus,
learning those dependencies (or semantic relationships) is at best difficult
and at worst impossible.

Graph-based models incorporate syntactic and semantic dependencies between
different code elements. Source code can be transformed to syntactic graphs
(Abstract Syntax Trees) and semantic graphs (Control Flow Graphs, Data Flow
Graphs, Program Dependency Graphs, and so on). Devign [?] uses Code Property
Graphs (CPGs) to build a vulnerability detection model as proposed by
Yamaguchi et al. [?]. Chakraborty et al. [?] also generate code property
graphs from source code to consider the syntax and the semantics of the code.
CPGs consider succinct information from the control-flow and data-flow graphs
in addition to the AST and the program dependency graphs. Each of these
elements offer additional context about the semantic structure of the code
that may be relevant for vulnerability detection.

Both token and graph-based models suffer from vocabulary explosion—the number
of possible identifiers (e.g., variable and function names) in the code can be
infinite. Some approaches replace tokens with abstract names [?; ?]. Other
techniques use word embedding tools (e.g., word2vec) to create vector
representations of every token. For instance, VulDeePecker [?] and SySeVR [?]
use word2vec to transform symbolic tokens into vectors. In contrast, Devign
[?] uses word2vec to transform pure code tokens to real vectors. Alon et al.
[?] proposed continuous distributed vector (or code embeddings) to represent
code. Code2vec aggregates an arbitrary sized snippet of code into a fixed-size
vector in a way that captures its semantics. Code functions are transformed in
groups of path-contexts, where each path-context represents a semantic
relationship between two code elements in a function.

CodeBERT [?] is a transformer-based model that represents given snippets of
source code in a distributed representation vector [?]. The non-sequential
nature of the architecture of the transformer encoder, being based on a simple
attention mechanism, is designed to address the problem of reasoning about
long sequences: each token is processed in parallel throughout the model.
CodeBERT was pre-trained on pairs of programming language and natural language
data. A pre-trained model produces distributed representations that can be
used in a variety of downstream tasks, on which the model itself can be
further fine-tuned.

This study evaluated how Code2vec—a model that considers syntactic and
semantic relationships in the code—fairs compared to other non-token based and
token-based models (specifically, CodeBERT) for vulnerability detection in
C/C++.

##  7 Conclusions

We applied Code2vec, a model for distributed code representations using AST
path-contexts, to the task of binary vulnerability detection. We evaluated
Code2vec on the Devign dataset as part of the CodeXGLUE benchmark. Our
experiments achieved an accuracy score that outperformed naive NLP-based
approaches and was equivalent to a simple transformer-based model that had not
been pre-trained on source code data. However, as expected, it was
outperformed by more expressive models that directly leverage features unique
to the source code. Additionally, we showed that smaller models such as
Code2vec have modest computational resource requirements compared to other
alternatives: when computational resources are scarce, the reduced training
time requirement and memory consumption of Code2vec on a labeled dataset of
source code functions may outweigh the loss in accuracy that results from its
lower expressiveness.

The performance of Code2vec for vulnerability detection could potentially be
improved through hyperparameter tuning, for example, by optimally choosing the
maximum length and width of path-contexts as well as the learning rate and
batch size. We plan to investigate these improvements in future work. Although
Code2vec is limited compared to state-of-the-art, it remains an attractive
choice for developers for code intelligence tasks such as vulnerability
detection, as it demonstrates comparable performance and can be more easily
integrated in CI pipelines than static analyzers or larger models.

## Acknowledgments

This work was supported in part by Fundação para a Ciencia e a Tecnologia
(FCT) under Grants CMU/TIC/0064/2019 (a project funded by the Carnegie Mellon
Portugal Program) and UIDB/50021/2020.

## References

  * [1] Jason Bau, Elie Bursztein, Divij Gupta, and John Mitchell.  State of the art: Automated black-box web application vulnerability testing.  In 2010 IEEE Symposium on Security and Privacy, pages 332–345, 2010\.
  * [2] Juha Kuusela.  Security testing in continuous integration processes.  Master’s thesis, Aalto University. School of Science, 2017.
  * [3] Fiorella Zampetti, Simone Scalabrino, Rocco Oliveto, Gerardo Canfora, and Massimiliano Di Penta.  How open source projects use static code analysis tools in continuous integration pipelines.  In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), pages 334–344, 2017.
  * [4] J. Zheng, L. Williams, N. Nagappan, W. Snipes, J.P. Hudepohl, and M.A. Vouk.  On the value of static analysis for fault detection in software.  IEEE Transactions on Software Engineering, 32(4):240–253, 2006\.
  * [5] Paul Anderson.  The use and limitations of static-analysis tools to improve software quality.  CrossTalk: The Journal of Defense Software Engineering, 21(6):18–21, 2008.
  * [6] Jacob A. Harer, Louis Y. Kim, Rebecca L. Russell, Onur Ozdemir, Leonard R. Kosta, Akshay Rangamani, Lei H. Hamilton, Gabriel I. Centeno, Jonathan R. Key, and Paul M. Ellingwood.  Automated software vulnerability detection with machine learning.  arXiv preprint arXiv:1803.04497, 2018.
  * [7] Guanjun Lin, Sheng Wen, Qing-Long Han, Jun Zhang, and Yang Xiang.  Software vulnerability detection using deep neural networks: A survey.  Proceedings of the IEEE, 108(10):1825–1848, 2020.
  * [8] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [9] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * [10] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever.  Improving language understanding by generative pre-training.  2018\.
  * [11] Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, and Daxin Jiang.  CodeBERT: A pre-trained model for programming and natural languages.  arXiv preprint arXiv:2002.08155, 2020.
  * [12] Luca Buratti, Saurabh Pujar, Mihaela Bornea, Scott McCarley, Yunhui Zheng, Gaetano Rossiello, Alessandro Morari, Jim Laredo, Veronika Thost, and Yufan Zhuang.  Exploring software naturalness through neural language models.  arXiv preprint arXiv:2006.12641, 2020.
  * [13] Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang.  Unified pre-training for program understanding and generation.  arXiv preprint arXiv:2103.06333, 2021.
  * [14] Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, and Duyu Tang.  CodeXGLUE: A machine learning benchmark dataset for code understanding and generation.  arXiv preprint arXiv:2102.04664, 2021.
  * [15] Nimit Sharad Sohoni, Christopher Richard Aberger, Megan Leszczynski, Jian Zhang, and Christopher Ré.  Low-memory neural network training: A technical report.  arXiv preprint arXiv:1904.10631, 2019.
  * [16] Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav.  code2vec: Learning distributed representations of code.  Proceedings of the ACM on Programming Languages, 3, January 2019\.
  * [17] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  RoBERTa: A robustly optimized BERT pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * [18] Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu.  Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks.  In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
  * [19] Vladimir Kovalenko, Egor Bogomolov, Timofey Bryksin, and Alberto Bacchelli.  PathMiner: a library for mining of path-based representations of code.  In Proceedings of the 16th International Conference on Mining Software Repositories, pages 13–17. IEEE Press, 2019.
  * [20] Alex Graves and Jürgen Schmidhuber.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures.  Neural Networks, 18(5):602–610, 2005.  IJCNN 2005.
  * [21] Yoon Kim.  Convolutional neural networks for sentence classification.  In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751, Doha, Qatar, October 2014. Association for Computational Linguistics.
  * [22] Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong.  Vuldeepecker: A deep learning-based system for vulnerability detection.  Proceedings 2018 Network and Distributed System Security Symposium, 2018.
  * [23] Zhen Li, Deqing Zou, Shouhuai Xu, Hai Jin, Yawei Zhu, and Zhaoxuan Chen.  Sysevr: A framework for using deep learning to detect software vulnerabilities.  IEEE Transactions on Dependable and Secure Computing, page 1, 2021\.
  * [24] Rebecca Russell, Louis Kim, Lei Hamilton, Tomo Lazovich, Jacob Harer, Onur Ozdemir, Paul Ellingwood, and Marc McConley.  Automated vulnerability detection in source code using deep representation learning.  In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 757–762, 2018.
  * [25] Fabian Yamaguchi, Nico Golde, Daniel Arp, and Konrad Rieck.  Modeling and discovering vulnerabilities with code property graphs.  In 2014 IEEE Symposium on Security and Privacy, pages 590–604, 2014\.
  * [26] Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray.  Deep learning based vulnerability detection: Are we there yet?  arXiv preprint arXiv:2009.07235, 2020.