# Towards Trustworthy AI Software Development Assistance

Daniel Maninger  0009-0005-0649-4958 Technische Universität DarmstadtHessian
Center for Artificial Intelligence (hessian.AI)DarmstadtGermany
<EMAIL_ADDRESS>,  Krishna Narasimhan  0000-0001-8004-3470 AI
Quality & Testing HubFrankfurt am MainGermany<EMAIL_ADDRESS>and  Mira Mezini  0000-0001-6563-7537 Technische Universität DarmstadtHessian
Center for Artificial Intelligence (hessian.AI)National Research Center for
Applied Cybersecurity ATHENEDarmstadtGermany<EMAIL_ADDRESS>
(2024; 14 September 2023; 21 Dezember 2023; 22 November 2023)

###### Abstract.

It is expected that in the near future, AI software development assistants
will play an important role in the software industry. However, current
software development assistants tend to be unreliable, often producing
incorrect, unsafe, or low-quality code. We seek to resolve these issues by
introducing a holistic architecture for constructing, training, and using
trustworthy AI software development assistants. In the center of the
architecture, there is a foundational LLM trained on datasets representative
of real-world coding scenarios and complex software architectures, and fine-
tuned on code quality criteria beyond correctness. The LLM will make use of
graph-based code representations for advanced semantic comprehension. We
envision a knowledge graph integrated into the system to provide up-to-date
background knowledge and to enable the assistant to provide appropriate
explanations. Finally, a modular framework for constrained decoding will
ensure that certain guarantees (e.g., for correctness and security) hold for
the generated code.

Software Development, AI Assistants, Large Language Models, Code Models, Code
Quality, Code Security

††copyright: rightsretained††journalyear: 2024††doi:
10.1145/3639476.3639770††conference: New Ideas and Emerging Results; April
14–20, 2024; Lisbon, Portugal††booktitle: New Ideas and Emerging Results
(ICSE-NIER’24), April 14–20, 2024, Lisbon, Portugal††isbn:
979-8-4007-0500-7/24/04††ccs: Software and its engineering Software notations
and tools††ccs: Computing methodologies Neural networks

##  1\. Introduction

Software development (SD) is a complex, expensive process (Boehm et al.,
2000). In times of ever-increasing digitalization of our society and scarcity
of IT talent, it makes sense to (partially) automate this process – the
popularity that AI-assisted code generation has gained since the appearance of
large language models (LLMs) indicates this need. At the same time, given that
software dominates our digitized society, guaranteeing the quality of the
generated code is paramount. Moreover, for the acceptance of AI software
development assistants, it is essential that they cover a broad range of
software engineering tasks and faithfully mimic existing well-understood and
proven SD methods and practices.

Although existing LLMs, when pre-trained or fine-tuned on code111For brevity,
we will use the term _code model_., have shown promise in handling coding-
related queries and synthesizing coherent-looking code, they exhibit
concerning issues. For instance, AIs like ChatGPT produce erroneous code
suggestions: On Stack Overflow, the large number of ChatGPT-generated
inaccurate answers led to a ban on its use. The moderators stated that “the
posting of answers created by ChatGPT […] is substantially harmful to the site
and to users who are asking questions and looking for correct
answers”222https://meta.stackoverflow.com/questions/421831/temporary-policy-
chatgpt-is-banned. A later study made similar observations (Kabir et al.,
2023).

Moreover, a study on GitHub Copilot revealed that 40% of the generated code
contained critical security vulnerabilities (Pearce et al., 2022). Further
emphasizing the dangers of such issues in the domain of software engineering
is an investigation that discovered that “participants who had access to the
AI assistant were more likely to introduce security vulnerabilities […], yet
also more likely to rate their insecure answers as secure” (Perry et al.,
2023). These studies highlight quality and security problems with current
industrial AI software development assistants.

To remedy these problems, we seek to establish the foundations of a next-
generation AI software development assistant system333For brevity, we will
also refer to it simply as _(AI SD) assistant_.. Given the software engineer’s
natural language queries, the assistant should suggest _high-quality_ code,
i.e., correct but also secure, readable, and otherwise compliant with best
practices. So far, code model research has focused almost exclusively on
correctness (e.g., (Feng et al., 2020; Guo et al., 2021; Kim et al., 2021; Le
et al., 2022; Li et al., 2022; Shiv and Quirk, 2019)).

Beyond that, the assistant should be able to serve the role of a virtual pair
programmer, explaining its suggestions, helping with debugging, etc., similar
to how a human would. The positive effects of pair programming practices on
code correctness and quality (Williams et al., 2000; Williams and Kessler,
2003; Nawrocki and Wojciechowski, 2001), as well as on collaborative learning
(Nadi et al., 2016), are well known. Despite this, pair programming has not
been widely adopted because working closely with a teammate is
challenging444https://martinfowler.com/articles/on-pair-
programming.html#Challenges. A virtual pair programmer would not cause such
problems. Moreover, we envision the system we build as an AI counterpart to
the traditional DevOps pipeline. Therefore, our assistant should be able to
support the software engineer in all phases, from software design to
deployment.

The listed objectives are highly ambitious and require addressing various
challenges. This paper is intended to outline a way forward for us and other
researchers trying to develop better AI software development assistants. To
this end, we start by identifying five key challenges:

  1. (1)

lack of representative datasets,

  2. (2)

difficulties capturing code structure and semantics,

  3. (3)

low code quality,

  4. (4)

insufficient explanations, and

  5. (5)

no guarantees on the results.

We then propose five solutions to address these challenges:

  1. (1)

curated real-world datasets,

  2. (2)

graph representations of code,

  3. (3)

fine-tuning through feedback from code analyses

  4. (4)

enriched code knowledge graphs, and

  5. (5)

constrained decoding.

Our considerations lead to a holistic multi-component architecture for AI
software development assistant systems built around a central code model. It
is illustrated in Figure 1.

Figure 1. High-level architecture of our envisioned AI software development
assistant. It consists of five main components: (1) A curated training
_dataset_ representing real-world coding patterns and software architectures.
(2) A foundational _code model_ that uses _graph representations_ for better
understanding of program semantics. (3) An RL-based _feedback mechanism_ to
fine-tune the model for improved code quality. (4) A semantically enriched
_code knowledge graph_ to help the assistant explain its code. (5) A modular
_constrained decoding_ framework on top of the model that prevents the
generation of undesired code.

##  2\. Approach

The first step towards improving current AI software development assistants is
to understand their limitations. Here, we elaborate on the five challenges we
found to be most relevant. For each of them, we propose a possible solution
together with an evaluation plan. All the described ideas contribute to the
overall goal of this vision: Creating a _trustworthy_ AI SD assistant that a
software engineer can use in real-world software projects without having to
worry about the correctness, quality, safety, and security of the generated
code.

###  2.1. Representative Datasets

#### Challenge

Popular code datasets555With closed-source commercial code models, the
datasets used are often unknown. such as
CodeSearchNet666https://github.com/github/CodeSearchNet (Husain et al., 2019)
and Py150K777https://www.sri.inf.ethz.ch/py150 consists of individual code
snippets with single functions. Such datasets do not represent real-world
software, typically exhibiting complex software patterns, interdependent class
hierarchies, project dependencies involving multiple interconnected files, and
further complexities. A study by Hellendoorn et al. (Hellendoorn et al., 2019)
revealed that the absence of these features in coding benchmarks contributed
to many shortcomings of code completion systems. We, therefore, consider the
lack of high-quality datasets that accurately represent real-world coding
patterns and complex software architectures a key challenge for training code
models. However, real-world code suitable for creating such datasets is
scarce. Moreover, annotating and curating code datasets requires significant
code quality and dataset management expertise.

#### Envisioned solution

To meet this challenge, we plan to compile and curate a comprehensive dataset
of code that accurately reflects common real-world coding patterns and
software structures. For this, we will build upon established state-of-the-art
datasets. To make these datasets representative, we will devise heuristics
rooted in general coding patterns observed in real-world coding practices.
Hellendoorn et al. introduced initial heuristics for the C# programming
language to distinguish real-world completions from synthetic ones, such as
length, type, and origin of (syntactic) tokens. We will expand on these
heuristics in a generalized manner for other popular languages, including
Java, Python, and C.

We will annotate our dataset with qualitative metrics to ensure we train with
high-quality code. We will rely on various techniques to accurately label the
dataset, including static code analysis tools, Stack Overflow discussions, and
GitHub issue trackers. While mining GitHub repositories, strategies must be
adopted to circumvent known pitfalls (Kalliamvakou et al., 2014). When
extracting labels from Stack Overflow, we will address challenges posed by
incomplete and unstructured code suggestions in the discussions by exploring
automated program repair (Goues et al., 2019), including pattern-based (Kim et
al., 2013) and neural (Gupta et al., 2017) approaches.

#### Evaluation

To evaluate the impact of the representative dataset, we will train code
models like CodeBert (Feng et al., 2020), AlphaCode (Li et al., 2022), and
TravTrans (Kim et al., 2021) on vanilla versions of CodeSearchNet.
Subsequently, we will curate a variant of CodeSearchNet that represents real-
world coding patterns and retrain the models on it. We then compare the
differently trained model variants with each other. Standard metrics such as
exact match score, statement-level accuracy, or BLEU (Papineni et al., 2002)
are of limited value here, because programming tasks can be solved in multiple
ways and different looking programs can be semantically equivalent (Ren et
al., 2020). Therefore, we will evaluate the model performance using CodeBLEU
(Ren et al., 2020), which also considers abstract syntax trees and data flow.
Additionally, we will explore the use of techniques like semantic parsing and
program analysis to estimate the semantic similarity of code segments.

###  2.2. Capturing Code Structure and Semantics

#### Challenge

In Section 1, we gave some examples of issues with state-of-the-art AI SD
assistants, particularly their shortcomings in generating correct and secure
code. Note that these are errors at the semantic level. How and to what extent
code models can capture and respect program semantics remains an open research
question888Earlier works have attempted to investigate this question for code
transformers trained on abstract syntax tree (AST) representations (Wan et
al., 2022). However, such studies are limited, as ASTs may not capture program
semantics very well, and the approaches cannot easily be adapted to new
models. . One conceivable factor is that current code models treat code as
formatted text and discard crucial semantic information such as control or
data flow (Chakraborty et al., 2022), implicit usage constraints of third-
party libraries, and domain-specific knowledge. It remains unclear what
representations of code and SD knowledge should be used by these models.

#### Envisioned solution

To quantify the problems with current textual code representations, we are
developing an analysis technique to assess how much of a program’s semantics
transformer models learn to capture. We do this by calculating their _semantic
precision and recall_ , i.e., how well the models’ attention maps match the
codes’ control and data flow graphs. We will also consider the _graph edit
distance_ (Gao et al., 2010) between them.

Further, since most semantic-rich representations of code are graph-based –
control flow graphs, data flow graphs, call graphs – code models may benefit
from native support of graph-based representations. To this end, we will
explore different approaches and investigate their impact on the quality of
suggested code: Firstly, we will work on flattening graph representations for
code and using them as training data for sequence-based transformers.
Positional encodings specialized for graph structures (Shiv and Quirk, 2019)
may be instrumental in teaching the model how to interpret its input sequences
correctly. Secondly, approaches related to graph attention networks
(Velickovic et al., 2018) and attention masking (McDonald and Chiang, 2021)
will be worth exploring, as they enable constraints on graph relations to be
injected directly into the transformer’s attention heads. Thirdly, we will
explore the potential of graph neural networks (Zhou et al., 2020) and graph
transformers (Yun et al., 2019) to operate directly on graph-based code
representations. We will investigate the trade-offs between all three
approaches. Potentially, this effort will lead to a foundational code model
with increased semantic awareness.

#### Evaluation

To evaluate the semantic awareness of our graph-based code models, we will use
the novel analysis technique described above. Additionally, to evaluate their
robustness, we randomly select code artifacts and make semantic-preserving
changes to them – renaming variables, translating for-statements into while-
statements, etc. In a robust semantic-aware model, such semantics-preserving
changes should typically not be reflected in large-scale changes to the
attention maps. We will also compare the performance of the semantic-aware
foundational code models we build with standard models like CodeBert,
TravTrans, and GraphCodeBert (Guo et al., 2021) on downstream tasks using the
metrics mentioned in Subsection 2.1.

###  2.3. Code Quality

#### Challenge

Human software engineers receive feedback on their code as they write it
through automated tools like compiler checks and linters, or from colleagues
during pair programming and code reviews. Such feedback loops can have a very
positive impact on code quality. For example, studies on pair programming
report fewer bugs, better readability, higher test coverage and passing rates,
and other benefits (Williams et al., 2000; Williams and Kessler, 2003;
Nawrocki and Wojciechowski, 2001). Code models, on the other hand, lack
similar mechanisms. They are typically optimized for accuracy, not for
qualitative criteria. We believe that a reliable AI SD assistant should
provide code that not only accurately reflects existing code patterns, but is
also of high quality according to well-established quality criteria and best
practices.

#### Envisioned solution

Inspired by the feedback loop described above, we propose an approach based on
reinforcement learning (RL) (Sutton and Barto, 1998) to fine-tune code models
for multiple code quality criteria. The approach shares ideas with
actor–critic RL (Konda and Tsitsiklis, 1999; Sutton et al., 1999), where, in
our case, the code model takes the actor’s role, and one or more code analysis
tools serve as critics. Each critic evaluates the code generated by the actor
and returns a token-wise reward value. The rewards are then aggregated (and
traded off against each other) using _utility functions_ known from multi-
objective RL (Hayes et al., 2022). Finally, the policy gradient is used to
update the actor model, e.g., through proximal policy optimization (PPO)
(Schulman et al., 2017).

While ideas with actor–critic RL have recently been explored for program
correctness (Le et al., 2022), our setting has a number of special
characteristics that are not systematically explored: Since there is no
universal metric for code quality, we resort to multiple critics, each
specializing in one or more qualitative aspects. Possible critics for general
best practices are linters and similar static analysis tools. To optimize
security aspects, specialized security checkers like CogniCrypt (Krüger et
al., 2017) or CryptoGuard (Rahaman et al., 2019) can be used. One could also
consider more subjective criteria such as readability metrics (Scalabrino et
al., 2018). In our setting, critics must provide individual rewards for each
token. This enables reward shaping and, thus, more efficient learning, as it
conveys to the actor which actions (i.e., generated tokens) were particularly
good or bad. Despite this, our setting has the benefit that we can generate a
complete program before applying the critics to it. This allows us to use a
wide variety of critics, even if they cannot handle partial (incomplete)
programs. Lastly, we must point out that multi-objective RL with many (i.e.,
four or more) objectives is considered a hard problem (Hayes et al., 2022). In
our case, however, the different code quality objectives can be assumed to be
mostly non-conflicting. Therefore, there is no need to find all Pareto-optimal
policies – one is sufficient – which allows us to use much more efficient
algorithms.

#### Evaluation

The effects of our code quality fine-tuning approach can be measured by a
direct comparison of variants of a code model before and after fine-tuning.
The same tools used as critics can be used to evaluate the code generated by
the two variants. In addition, the code should be audited and rated by humans
experts. For most code quality criteria, any coding benchmark is suitable,
while specialized benchmarks like LLMSecEval (Tony et al., 2023) can be used
to evaluate safety and security criteria.

###  2.4. Explainability

#### Challenge

Current code models are not designed to base their answers on concrete
background knowledge and use that knowledge to account for their answers to
the user. While instruction fine-tuned (code) models (Zhang et al., 2023)
often add explanations to their answers, these answers are the result of
purely statistical computations and may be hallucinated. For instance, a
recent study by Kabir et al. (Kabir et al., 2023) found that ChatGPT answered
Stack Overflow questions incorrectly half of the time. In our view, the issue
of how to enable LLMs to provide correct and appropriate explanations for code
remains unresolved999A closely related unresolved problem is how to keep a
code model in sync with the ever-evolving correct coding patterns and best
practices. This is particularly important when writing safety or security-
critical software, where deprecated features should never be used. Given that
expensive regular retraining is not desirable, explicit access to (up-to-date)
background knowledge is required here as well. Hence, one approach may address
both of these problems at once..

#### Envisioned solution

For this challenge, we plan to investigate how to integrate _code knowledge
graphs_ (CKGs) into our AI SD assistant. This way, we want to give the
assistant access to a rich, easily updatable source of background knowledge
that it can ground its generated code and explanations in. A potential subject
for our investigation is IBM’s
GraphGen4Code101010https://wala.github.io/graph4code/ (Abdelaziz et al.,
2021). This CKG is a set of triplets describing relationships (edges) between
entities (nodes). For a given piece of code, the corresponding CKG contains
nodes for each statement, with edges signifying control and data flow.
Additionally, GraphGen4Code has established initial semantic links between
code fragments and corresponding forum discussions on Stack Overflow. However,
the expressiveness of these name-based links is rather superficial.

To improve the expressiveness of the relationships between code and knowledge
sources, we propose to enrich GraphGen4Code with information about the task to
which a particular code description or forum post relates. This information
can be obtained using intent classification. Moreover, while Stack Overflow is
a valuable source of discussions about and fixes to programming issues, the
GitHub issue tracker can be another equally important source of such
information. We thus propose to add links between code segments and GitHub
issues to the CKG, along with additional information, such as whether a pull
request successfully fixed the corresponding issue.

How to integrate the CKG into the assistant remains to be determined. One
option is to augment the code model to query the CKG during the generation of
its answer – several papers have already demonstrated how to teach LLMs to use
external tools or APIs (Mialon et al., 2023). Alternatively, in a post-
processing step, the CKG could be searched for code segments similar to the
generated one. Then, the model could do a second pass over the generated code,
using the retrieved information to “decorate” the generated code with
explanations. In order to find relevant code samples in the CKG, a separate
retriever model could be used (Lu et al., 2022).

#### Evaluation

To evaluate the extended CKG’s ability to be integrated into the assistant and
provide accurate and appropriate explanations to the software engineer, we
will utilize evaluation protocols similar to those used to evaluate the
original GraphGen4Code. We will involve human annotators and design an
agreement metric to rank the quality of the assistant’s suggestions and
explanations. We will also evaluate the computing cost of enhancing large
CKG’s with complex semantic links and new sources of information such as
GitHub issues.

###  2.5. Controlled Code Generation

#### Challenge

All measures presented so far serve to enhance the capabilities of the AI SD
assistant. However, due to the statistical nature of LLMs, even the best model
may occasionally produce bad (e.g., incorrect, insecure, or low-quality)
results. This lack of guarantees is a major hindrance to the trustworthiness
of AI SD assistants.

#### Envisioned solution

To provide the desired guarantees, we propose to equip the assistant with
_constrained (a.k.a. guided) decoding_ (e.g., (Poesia et al., 2022)) – a
recently popular extension of the decoding algorithms of causal LLMs that
enforces compliance with certain rules for the generated text. At each
generation step, all tokens that would lead to a violation of the rules are
prevented from being generated by setting their probability to zero before
applying the regular decoding strategy. The underlying rules can, e.g., be
defined through regular expressions or grammars and the implementations can be
very efficient (Willard and Louf, 2023). This approach has already been
applied to coding tasks, where it could, among other things, ensure syntactic
correctness (Poesia et al., 2022) and prevent hallucinations (Agrawal et al.,
2023). Models equipped with constrained decoding often outperform larger
models (Agrawal et al., 2023; Poesia et al., 2022; Willard and Louf, 2023).

Since constrained decoding rules can be implemented and applied independently
of each other, we envision a modular constrained decoding framework on top of
the central code model. The modularity allows the user to select a ruleset
appropriate for the current programming language and domain. There is a wide
variety of rules that could be implemented. Constrained decoding is, however,
no replacement for a capable code model for three reasons: rules must be
checkable given only a program _prefix_ ; for efficiency reasons, they should
be _incremental_ , i.e., not require reanalysis of the program at each
generation step; and, depending on the complexity of the rules, the
constraints may be _unsound_ and/or _incomplete_. Still, the lower bound on
code quality, and thus the trustworthiness of the assistant, that constrained
decoding provides is very valuable.

#### Evaluation

Measuring how effective constrained decoding is in preventing undesired
outputs is easy: First, we perform _unconstrained_ decoding on a series of
prompts. Then we iterate over the output sequences using the constrained
decoding framework and count in how many cases at least one of the rules was
violated (and the constrained decoding would have intervened). This could also
be insightful as it reveals typical errors made by unconstrained models. On
the other hand, rules can be overly strict and guide the decoding into “dead
ends” where the model can only terminate the generation or produce degenerate
(i.e., valid but useless) results. How common this is compared to
unconstrained decoding can be determined via standard benchmarks. The outcome
of the described evaluations will of course highly depend on the used
foundation model and ruleset.

##  3\. Future Plans

The vision presented in this paper encompasses multiple ideas, all working
towards creating a trustworthy AI software development assistant. Despite
this, each idea is independent enough to form its own line of research.
Therefore, they are worked on in parallel by different subgroups of
researchers and lead to several full papers. At the current state, all
components are conceptualized but still need to be implemented and evaluated
as described above. Combining all outcomes to build the described programming
assistant is a highly ambitious goal and will take several years to realize.

##  4\. Conclusion

In this paper, we shared our understanding of the shortcomings of current code
models and presented our vision of how they can be overcome. We considered
aspects such as representative datasets, semantic code representations, code
quality, explainability, and guarantees.

###### Acknowledgements.

Funded by the Hessian Ministry of Higher Education, Research, Science and the
Arts within the cluster project The Third Wave of Artificial Intelligence
(3AI).

## References

  * (1)
  * Abdelaziz et al. (2021) Ibrahim Abdelaziz, Julian Dolby, Jamie P. McCusker, and Kavitha Srinivas. 2021.  A Toolkit for Generating Code Knowledge Graphs. In _K-CAP ’21: Knowledge Capture Conference, Virtual Event, USA, December 2-3, 2021_ , Anna Lisa Gentile and Rafael Gonçalves (Eds.). ACM, 137–144.
  * Agrawal et al. (2023) Lakshya A Agrawal, Aditya Kanade, Navin Goyal, Shuvendu K. Lahiri, and Sriram K. Rajamani. 2023.  Guiding Language Models of Code with Global Context using Monitors.  _CoRR_ abs/2306.10763 (2023).
  * Boehm et al. (2000) Barry W. Boehm, Chris Abts, and Sunita Chulani. 2000\.  Software development cost estimation approaches - A survey.  _Ann. Softw. Eng._ 10 (2000), 177–205.
  * Chakraborty et al. (2022) Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2022.  Deep Learning Based Vulnerability Detection: Are We There Yet?  _IEEE Trans. Software Eng._ 48, 9 (2022), 3280–3296.
  * Feng et al. (2020) Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020.  CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In _Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020_ _(Findings of ACL, Vol. EMNLP 2020)_ , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 1536–1547.
  * Gao et al. (2010) Xinbo Gao, Bing Xiao, Dacheng Tao, and Xuelong Li. 2010\.  A survey of graph edit distance.  _Pattern Anal. Appl._ 13, 1 (2010), 113–129.
  * Goues et al. (2019) Claire Le Goues, Michael Pradel, and Abhik Roychoudhury. 2019\.  Automated program repair.  _Commun. ACM_ 62, 12 (2019), 56–65.
  * Guo et al. (2021) Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin B. Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021\.  GraphCodeBERT: Pre-training Code Representations with Data Flow. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Gupta et al. (2017) Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish K. Shevade. 2017\.  DeepFix: Fixing Common C Language Errors by Deep Learning. In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA_ , Satinder Singh and Shaul Markovitch (Eds.). AAAI Press, 1345–1351.
  * Hayes et al. (2022) Conor F. Hayes, Roxana Radulescu, Eugenio Bargiacchi, Johan Källström, Matthew Macfarlane, Mathieu Reymond, Timothy Verstraeten, Luisa M. Zintgraf, Richard Dazeley, Fredrik Heintz, Enda Howley, Athirai A. Irissappane, Patrick Mannion, Ann Nowé, Gabriel de Oliveira Ramos, Marcello Restelli, Peter Vamplew, and Diederik M. Roijers. 2022\.  A practical guide to multi-objective reinforcement learning and planning.  _Auton. Agents Multi Agent Syst._ 36, 1 (2022), 26\.
  * Hellendoorn et al. (2019) Vincent J. Hellendoorn, Sebastian Proksch, Harald C. Gall, and Alberto Bacchelli. 2019.  When code completion fails: a case study on real-world completions. In _Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019_ , Joanne M. Atlee, Tevfik Bultan, and Jon Whittle (Eds.). IEEE / ACM, 960–970.
  * Husain et al. (2019) Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019.  CodeSearchNet Challenge: Evaluating the State of Semantic Code Search.  _CoRR_ abs/1909.09436 (2019).
  * Kabir et al. (2023) Samia Kabir, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2023.  Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions.  _CoRR_ abs/2308.02312 (2023).
  * Kalliamvakou et al. (2014) Eirini Kalliamvakou, Georgios Gousios, Kelly Blincoe, Leif Singer, Daniel M. Germán, and Daniela E. Damian. 2014.  The promises and perils of mining GitHub. In _11th Working Conference on Mining Software Repositories, MSR 2014, Proceedings, May 31 - June 1, 2014, Hyderabad, India_ , Premkumar T. Devanbu, Sung Kim, and Martin Pinzger (Eds.). ACM, 92–101.
  * Kim et al. (2013) Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim. 2013\.  Automatic patch generation learned from human-written patches. In _35th International Conference on Software Engineering, ICSE ’13, San Francisco, CA, USA, May 18-26, 2013_ , David Notkin, Betty H. C. Cheng, and Klaus Pohl (Eds.). IEEE Computer Society, 802–811.
  * Kim et al. (2021) Seohyun Kim, Jinman Zhao, Yuchi Tian, and Satish Chandra. 2021\.  Code Prediction by Feeding Trees to Transformers. In _43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, 22-30 May 2021_. IEEE, 150–162.
  * Konda and Tsitsiklis (1999) Vijay R. Konda and John N. Tsitsiklis. 1999.  Actor-Critic Algorithms. In _Advances in Neural Information Processing Systems 12, [NIPS Conference, Denver, Colorado, USA, November 29 - December 4, 1999]_ , Sara A. Solla, Todd K. Leen, and Klaus-Robert Müller (Eds.). The MIT Press, 1008–1014.
  * Krüger et al. (2017) Stefan Krüger, Sarah Nadi, Michael Reif, Karim Ali, Mira Mezini, Eric Bodden, Florian Göpfert, Felix Günther, Christian Weinert, Daniel Demmler, and Ram Kamath. 2017\.  CogniCrypt: supporting developers in using cryptography. In _Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering, ASE 2017, Urbana, IL, USA, October 30 - November 03, 2017_ , Grigore Rosu, Massimiliano Di Penta, and Tien N. Nguyen (Eds.). IEEE Computer Society, 931–936.
  * Le et al. (2022) Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu-Hong Hoi. 2022\.  CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning. In _NeurIPS_.
  * Li et al. (2022) Yujia Li, David H. Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. 2022.  Competition-Level Code Generation with AlphaCode.  _CoRR_ abs/2203.07814 (2022).
  * Lu et al. (2022) Shuai Lu, Nan Duan, Hojae Han, Daya Guo, Seung-won Hwang, and Alexey Svyatkovskiy. 2022.  ReACC: A Retrieval-Augmented Code Completion Framework. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022_ , Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 6227–6240.
  * McDonald and Chiang (2021) Colin McDonald and David Chiang. 2021.  Syntax-Based Attention Masking for Neural Machine Translation. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, NAACL-HLT 2021, Online, June 6-11, 2021_ , Esin Durmus, Vivek Gupta, Nelson Liu, Nanyun Peng, and Yu Su (Eds.). Association for Computational Linguistics, 47–52.
  * Mialon et al. (2023) Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. 2023.  Augmented Language Models: a Survey.  _CoRR_ abs/2302.07842 (2023).
  * Nadi et al. (2016) Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden. 2016.  Jumping through hoops: why do Java developers struggle with cryptography APIs?. In _Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016_ , Laura K. Dillon, Willem Visser, and Laurie A. Williams (Eds.). ACM, 935–946.
  * Nawrocki and Wojciechowski (2001) Jerzy Nawrocki and Adam Wojciechowski. 2001.  Experimental evaluation of pair programming.  _European Software Control and Metrics (Escom)_ (2001), 99–101.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a Method for Automatic Evaluation of Machine Translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA_. ACL, 311–318.
  * Pearce et al. (2022) Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2022\.  Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions. In _43rd IEEE Symposium on Security and Privacy, SP 2022, San Francisco, CA, USA, May 22-26, 2022_. IEEE, 754–768.
  * Perry et al. (2023) Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2023.  Do Users Write More Insecure Code with AI Assistants?. In _Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, CCS 2023, Copenhagen, Denmark, November 26-30, 2023_ , Weizhi Meng, Christian Damsgaard Jensen, Cas Cremers, and Engin Kirda (Eds.). ACM, 2785–2799.
  * Poesia et al. (2022) Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022.  Synchromesh: Reliable Code Generation from Pre-trained Language Models. In _The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022_. OpenReview.net.
  * Rahaman et al. (2019) Sazzadur Rahaman, Ya Xiao, Sharmin Afrose, Fahad Shaon, Ke Tian, Miles Frantz, Murat Kantarcioglu, and Danfeng (Daphne) Yao. 2019.  CryptoGuard: High Precision Detection of Cryptographic Vulnerabilities in Massive-sized Java Projects. In _Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK, November 11-15, 2019_ , Lorenzo Cavallaro, Johannes Kinder, XiaoFeng Wang, and Jonathan Katz (Eds.). ACM, 2455–2472.
  * Ren et al. (2020) Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020\.  CodeBLEU: a Method for Automatic Evaluation of Code Synthesis.  _CoRR_ abs/2009.10297 (2020).
  * Scalabrino et al. (2018) Simone Scalabrino, Mario Linares-Vásquez, Rocco Oliveto, and Denys Poshyvanyk. 2018.  A comprehensive model for code readability.  _J. Softw. Evol. Process._ 30, 6 (2018).
  * Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017.  Proximal Policy Optimization Algorithms.  _CoRR_ abs/1707.06347 (2017).
  * Shiv and Quirk (2019) Vighnesh Leonardo Shiv and Chris Quirk. 2019.  Novel positional encodings to enable tree-based transformers. In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 12058–12068.
  * Sutton and Barto (1998) Richard S. Sutton and Andrew G. Barto. 1998.  _Reinforcement learning - an introduction_.  MIT Press.
  * Sutton et al. (1999) Richard S. Sutton, David A. McAllester, Satinder Singh, and Yishay Mansour. 1999.  Policy Gradient Methods for Reinforcement Learning with Function Approximation. In _Advances in Neural Information Processing Systems 12, [NIPS Conference, Denver, Colorado, USA, November 29 - December 4, 1999]_ , Sara A. Solla, Todd K. Leen, and Klaus-Robert Müller (Eds.). The MIT Press, 1057–1063.
  * Tony et al. (2023) Catherine Tony, Markus Mutas, Nicolás E. Díaz Ferreyra, and Riccardo Scandariato. 2023.  LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations. In _20th IEEE/ACM International Conference on Mining Software Repositories, MSR 2023, Melbourne, Australia, May 15-16, 2023_. IEEE, 588–592.
  * Velickovic et al. (2018) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018.  Graph Attention Networks. In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.
  * Wan et al. (2022) Yao Wan, Wei Zhao, Hongyu Zhang, Yulei Sui, Guandong Xu, and Hai Jin. 2022\.  What Do They Capture? - A Structural Analysis of Pre-Trained Language Models for Source Code. In _44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022_. ACM, 2377–2388.
  * Willard and Louf (2023) Brandon T. Willard and Rémi Louf. 2023.  Efficient Guided Generation for Large Language Models.  _CoRR_ abs/2307.09702 (2023).
  * Williams and Kessler (2003) Laurie A. Williams and Robert R. Kessler. 2003.  _Pair Programming Illuminated_.  Addison Wesley.
  * Williams et al. (2000) Laurie A. Williams, Robert R. Kessler, Ward Cunningham, and Ron Jeffries. 2000.  Strengthening the Case for Pair Programming.  _IEEE Softw._ 17, 4 (2000), 19–25.
  * Yun et al. (2019) Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo J. Kim. 2019.  Graph Transformer Networks. In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 11960–11970.
  * Zhang et al. (2023) Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2023.  Instruction Tuning for Large Language Models: A Survey.  _CoRR_ abs/2308.10792 (2023).
  * Zhou et al. (2020) Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020.  Graph neural networks: A review of methods and applications.  _AI Open_ 1 (2020), 57–81.