# Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A
Study using the Multilingual Defining Issues Test

Aditi Khandelwal1 Utkarsh Agarwal11110.4cm Kumar Tanmay11110.4cm Monojit
Choudhury2
$1$Microsoft Corporation
$2$MBZUAI
{t-aditikh, t-utagarwal<EMAIL_ADDRESS><EMAIL_ADDRESS>Equal contribution Work done while at
Microsoft.

###### Abstract

This paper explores the moral judgment and moral reasoning abilities exhibited
by Large Language Models (LLMs) across languages through the Defining Issues
Test. It iswe a well known fact that moral judgment depends on the language in
which the question is asked Costa et al. (2014). We extend the work of Tanmay
et al. (2023) beyond English, to 5 new languages (Chinese, Hindi, Russian,
Spanish and Swahili), and probe three LLMs – ChatGPT, GPT-4 and Llama2Chat-70B
– that shows substantial multilingual text processing and generation
abilities. Our study shows that the moral reasoning ability for all models, as
indicated by the post-conventional score, is substantially inferior for Hindi
and Swahili, compared to Spanish, Russian, Chinese and English, while there is
no clear trend for the performance of the latter four languages. The moral
judgments too vary considerably by the language.

Do Moral Judgment and Reasoning Capability of LLMs Change with Language? A
Study using the Multilingual Defining Issues Test

##  1 Introduction

In a recent work, Tanmay et al. (2023) used the Defining Issues Test (DIT)
Rest (1986), a psychological assessment tool based on Kohlberg’s Cognitive
Moral Development (CMD) Kohlberg (1958), to evaluate the moral reasoning
capabilities of large language models (LLMs) such as GPT-4, ChatGPT,
Llama2Chat-70B and PaLM-2. The DIT presents a moral dilemma along with 12
statements on ethical considerations and asks the respondent (in our case, the
LLM) to rank them in order of importance for resolving the dilemma. The test
outcome is a set of scores that indicate the respondent’s moral development
stage. According to this study Tanmay et al. (2023), GPT-4 was found to have
the best moral reasoning capability, equivalent to that of a graduate student,
while the three other models exhibited a moral reasoning ability that is at
par with an average adult.

Although interesting, the study was limited to English, even though many of
the models studied were multilingual. On the other hand, it is known that, for
humans, moral judgment often depends on the language in which the dilemma is
presented Costa et al. (2014). Language is a powerful tool that shapes our
thoughts, beliefs and actions. It can also affect how we perceive and resolve
moral dilemmas. Research in moral psychology has shown that people are more
likely to endorse utilitarian choices (such as sacrificing one person to save
five) when they read a dilemma in a foreign language (L2) than in their native
language (L1) Circi et al. (2021); Corey et al. (2017). This suggests that
language can modulate our emotional and cognitive responses to moral
situations.

To what extent does the moral judgment and reasoning capability of LLMs depend
on the language in which the question is asked, and what are the factors
responsible for the differences across languages, if any? In this paper, we
extend the DIT-based study by Tanmay et al. (2023) to five languages –
Spanish, Russian, Chinese, Hindi and Swahili. We study three popular LLMs -
GPT-4 OpenAI (2023), ChatGPT Schulman et al. (2022) and Llama2Chat-70B Touvron
et al. (2023), by probing them with the dilemmas and the moral considerations
separately for each language. We prompt the model to provide a resolution to
the dilemma and the list of top 4 most important moral considerations. The
responses are then used to compute the moral staging scores of the LLMs for
different languages.

Some of the salient observations of this study are: (1) GPT-4 has the best
multilingual moral reasoning capability with minimal difference in moral
judgment and staging scores across languages, while for LLama2Chat-70B and
ChatGPT the performance varies widely; (2) For all models, we observe superior
moral reasoning abilities for English and Spanish followed by Russian,
Chinese, Swahili and Hindi (in descending order of performance). Performance
in Hindi for ChatGPT and LLama2Chat-70B is no better than a random baseline.
(3) Despite high moral staging score for both English and Russian, we find
significant differences in moral judgment for these two languages, while the
judgments for English, Chinese and Spanish tend to agree more often.

While the difference in moral reasoning abilities across languages seem
correlated to the amount of resources available or used for training the
models, the reason behind the differences and similarities in the moral
judgments across the high resource languages (i.e., Chinese, English, Russian
and Spanish) is not obvious. We speculate it to be reflective of the values of
the societies where these languages are spoken, but also propose alternative
hypotheses.

Apart from being the first multilingual study of moral reasoning ability of
LLMs in the framework of Kohlberg’s CMD model, one key contribution of this
work is the creation of multilingual versions of the moral dilemmas presented
in DIT Rest (1986) and Tanmay et al. (2023). We will publicly share these
datasets, subject to permissions from the original authors.

##  2 Background: Moral Psychology and Ethics of NLP

Morality, the study of right and wrong, has long been a central topic in
philosophy Gert and Gert (2002). The Cognitive Moral Development (CMD) model
by Lawrence Kohlberg 1981 is a prominent theory that categorizes moral
development into three levels: pre-conventional, conventional, and post-
conventional morality. The Defining Issues Test (DIT) by James Rest 1979
measures moral reasoning abilities using moral dilemmas, providing insights
into ethical decision-making. This tool has been widely used for over three
decades, providing insights into ethical decision-making processes Rest et al.
(1994).

###  2.1 Defining Issues Test

DIT consists of several moral dilemmas. As an illustration, consider Timmy’s
Dilemma111DIT is behind a paywall, and hence, we cannot share the actual
dilemmas publicly. Therefore, we use this dilemma proposed by Tanmay et al.
(2023) as our running example: Timmy is a software engineer, working on a
crucial project that supports millions of customers. He discovers a bug in the
deployed system, which, if not fixed immediately, could put the privacy of
many customers at risk. Only Timmy knows about this bug and how to fix it.
However, Timmy’s best friend is getting married, and Timmy has promised to
attend and officiate the ceremony. If he decides to fix the bug now, he will
have to miss the wedding. Should Timmy go for the wedding (option 1), or fix
the bug first (option 3)? Or maybe it is simply not possible to decide (option
2).

In DIT, first, the respondent is asked to resolve such dilemmas that pit moral
values (in Timmy’s case between as professional vs. personal commitments)
against each other. The resolution is called the moral judgment offered by the
respondent. Then the respondent is presented with 12 moral consideration
statements. For instance, “Will Timmy get fired by his organization if he
doesn’t fix the bug?”, or “Should Timmy act according to his conscience and
moral values of loyalty towards a friend, and attend the wedding?” They are
asked to choose the 4 most important considerations (ranked by importance)
that helped them arrive at the moral judgment. In other words, the respondent
has to provide a moral reasoning for the judgment made. Each statement is
assigned to a specific moral development stage of the CMD model. A set of
moral development scores are then computed based on the response, which is
explained in detail in Section 3.4. Note that some statements are irrelevant
or against the conventions of society, which are ignored during the analysis
but can inform us about the attentiveness of the respondent.

###  2.2 Moral Judgment vs. Moral Reasoning

There is a long standing debate in moral philosophy and psychology on what
factors influence moral judgments Haidt (2001). While prominent philosophers
including Plato, Kant and Kohlberg have argued in favor of deductive reasoning
(not necessarily limited to pure logic) as the underlying mechanism, recent
research in psychology and neuroscience shows that in most cases people
intuitively arrive at a moral judgment and then use post-hoc reasoning to
rationalize it or explain/justify their position or to influence others in a
social setting (see Greene and Haidt (2002) for a survey). In this sense,
moral judgments are similar to aesthetic judgments rather than logical
deductions. It also explains why policy-makers often decide in favor of wrong
and unfair policies despite availability of clear evidence against those.

Therefore, DIT as well as its very foundation, Kohlberg’s CMD has been
criticized for over-emphasis on moral reasoning over moral intuitions Dien
(1982); Snarey (1985); Bebeau and Brabeck (1987); Haidt (2001). However, it
will be interesting to test the moral intuition vs. reasoning hypothesis for
LLMs, and what the alignment (or if we may say, “moral intuition”) of the
popular models are Yao et al. (2023). However, it will not only be interesting
to test the moral intuition vs. reasoning hypothesis for LLMs, and what the
alignment (or if we may say, “moral intuition”) of the popular models are Yao
et al. (2023), but also to explore how altering language affects moral
reasoning capabilities, assuming the value of analyzing such reasoning has
already been well argued for and established by previous work Rao et al.
(2023).

###  2.3 Language and Morality

Recent research Costa et al. (2014); Hayakawa et al. (2017); Corey et al.
(2017) reveals an intriguing connection between moral judgment and the
”Foreign-Language Effect”, that individuals tend to make more utilitarian
choices when faced with moral dilemmas presented in a foreign language (L2),
as opposed to their native tongue (L1). This shift appears to be linked to
reduced emotional responsiveness when using a foreign language, leading to a
diminished influence of emotions on moral judgments. Čavar and Tytus (2018)
also shows how a higher proficiency and a higher degree of acculturation in L2
may reduce utilitarianism in the L2 condition. This suggests that linguistic
factors can significantly influence moral decision-making, impacting a
substantial number of individuals. There are more complex interactions among
dilemma type, emotional arousal, and the language in bilingual individuals’
moral decision making process Chan et al. (2016).

###  2.4 Current Approaches to Ethics of LLMs

AI alignment aims to ensure AI systems align with human goals and ethics Piper
(Oct 15, 2020). Several work provide ethical frameworks Araque et al. (2020),
guidelines, and datasets Hoover et al. (2020); Trager et al. (2022); Alshomary
et al. (2022) for training and evaluating LLMs in ethical considerations and
societal norms Hendrycks et al. (2023). However, they may suffer from bias
based on annotator backgrounds Olteanu et al. (2019). Recent research
emphasizes in-context learning and supervised tuning to align LLMs with
ethical principles Zhou et al. (2023); Jiang et al. (2021); Rao et al. (2023).
These methods accommodate diverse ethical views that are essential given the
multifaceted nature of ethics. Tanmay et al. (2023) introduce an ethical
framework utilizing the Defining Issues Test to assess the ethical reasoning
capabilities of LLMs. The authors assessed the models performance with moral
dilemmas in English. To expand upon this work, our research delves deeper into
the performance of these models when confronted with moral dilemmas in a
multilingual context. This investigation aims to unveil how these LLMs respond
to the same scenarios in different languages, shedding light on their cross-
linguistic ethical reasoning capabilities.

###  2.5 Performance of LLMs across Languages

LLMs demonstrate impressive multilingual capability in natural language
processing tasks, but their proficiency varies across languages Zhao et al.
(2023). While their training data is primarily in English, it includes data
from other languages Brown et al. (2020); Chowdhery et al. (2022); Zhang et
al. (2022); Zeng et al. (2022). Despite their capabilities, the vast number of
languages worldwide, most of which are low-resource, presents a challenge.
LLMs still encounter difficulties with non-English languages, particularly in
low-resource settings Bang et al. (2023); Jiao et al. (2023); Hendy et al.
(2023); Zhu et al. (2023). Many studies have shown how the multilingual
performances of the LLMs can be improved using in-context learning and
carefully designed prompts Huang et al. (2023); Nguyen et al. (2023). Ahuja et
al. (2023) and Wang et al. (2023) report experiments for benchmarking the
multilingual capabilities of LLMs in various NLP tasks, such as Machine
Translation, Natural Language Inference, Sentiment Analysis, Text
Summarization, Named Entity Recognition, and Natural Language Generation, and
conclude that LLMs do not perform well for most but a few high resource
languages. Kovač et al. (2023) show that LLMs exhibit varying context-
dependent values and personality traits across perspectives, contrasting with
humans, who typically maintain more consistent values and traits across
contexts Schwartz (2012); Graham et al. (2013).

Existing research on multilingual LLMs has primarily focused on technical
capabilities, neglecting the exploration of their moral reasoning in diverse
linguistic and cultural contexts. This underscores the importance of probing
into the ethical dimensions of multilingual LLMs, given their significant
impact on various real-life applications and domains.

##  3 Experiments

In this section, we provide an overview of our experimental setup, datasets,
the language models (LLMs) that were studied, the structure of the prompts,
and the metrics employed. Our prompts to the LLMs include a moral dilemma
scenario, accompanied by a set of 12 ethical considerations and three
subsequent questions. By analyzing the responses to these questions, we
calculate the P-score as well as individual stage scores for each LLM which we
explain in Section 3.4.

###  3.1 Dataset and Prompt

We use the five dilemmas from DIT-1222Obtained the dataset by purchasing from
The University of Alabama through the official website:
https://ethicaldevelopment.ua.edu/ordering-information.html (Heinz, Newspaper,
Webster, Student, Prisoner) and four dilemmas introduced by Tanmay et al.
(2023). We used the Google Translation API to translate all these dilemmas
into six different languages: Hindi, Spanish, Swahili, Russian, Chinese, and
Arabic. To ensure the quality of translations, we asked native speakers of
Swahili, Hindi, Chinese, Russian, and Spanish to verify them. They suggested
some minor stylistic changes for 1-2 words per dilemma, but they confirmed
that the meaning was preserved. We also back-translated them into English to
check if the meaning remained consistent. Our choice was guided by our aim to
include diverse languages across three dimensions: (a) the amount of resource
available – Spanish, Chinese (high) to Hindi (medium) and Swahili (low); (b)
the script used - Spanish and Swahili use the Latin script, while Hindi,
Russian, Arabic, and Chinese employ non-Latin scripts, and (c) the cultural
context of the L1 speakers of the languages – Hindi and Swahili from Global
South representing traditional value-based cultures, Russian for orthodox
Europe, Spain for Catholic Europe and Chinese for Confucian system of values
(based on World Value Survey by Inglehart and Welzel (2010)). We followed the
same process as described in Tanmay et al. (2023) for the prompt, translating
it using the Google API and verifying the translations using the same
technique mentioned above. The prompt structure can be found in Figure 5 in
the Appendix.

###  3.2 Experimental Setup

We examined three of the most prominent LLMs with multilingual capabilities
Wang et al. (2023): GPT-4 (size undisclosed) OpenAI (2023), ChatGPT with 175
billion parameters Schulman et al. (2022), and Llama2-Chat with 70 billion
parameters Touvron et al. (2023). We applied the same shuffling strategy,
again as described by Tanmay et al. (2023), in resolving dilemmas by selecting
one of the three options (O1, O2, and O3) that is 6 permutations of options
and considering 8 distinct permutations out of the possible 12 statements (out
of 12! possibilities), resulting in a total of 48 permutations of prompts per
dilemma per language.

Throughout all our experiments, we set the temperature to 0, a presence
penalty of 1, and a top probabilities value of 0.95. Furthermore, we specified
a maximum token length of 2000 for English, Spanish, Chinese, Swahili, and
Russian, while for Hindi, we set a maximum token length of 4000, as it
requires a more tokens due to higher fertility of the tokenizer.

###  3.3 Method

We provide the translated prompt to the model and translate the response to
English using Google Translate API. Then we extract the responses of the three
questions posed in the DIT from the translated English response. We manually
check the answers for quality and find that for Arabic, the responses for
ChatGPT and Llama2Chat were getting truncated because of running out of
maximum token length of 4000. So we had to leave out Arabic from the rest of
our experiments. Hindi was excluded from our experiments with Llama2Chat
because limited context length of 4k token.

###  3.4 Metrics

DIT assesses three separate and developmentally ordered moral schemas Rest et
al. (1999). These schemas are identified as the Personal Interests schema,
which combines elements from Kohlberg’s Stages 2 and 3; the Maintaining Norms
schema, derived from Kohlberg’s Stage 4; and the Post-conventional schema,
which draws from Kohlberg’s Stages 5 and 6. The Post-conventional schema is
equivalent to the original summary index known as the P-score.

The Personal Interest schema score reflects an individual’s tendency to make
moral judgments based on their personal interests, desires, or self-benefit. A
higher score in this context suggests that a person is more inclined to
prioritize their own interests when making moral decisions. Maintaining norms
score measures a person’s commitment to upholding societal norms and rules in
their moral judgments. A higher score in this category indicates a greater
emphasis on adhering to established norms and societal expectations when
making ethical decisions. Post-conventionality score/$p_{score}$ gauges a
person’s level of moral development, reflecting their inclination to make
moral judgments based on advanced moral principles and ethical reasoning. A
higher score in this category signifies a commitment to abstract ethical
principles, justice, individual rights, and ethical values, transcending
conventional societal norms.

In summary, the Personal Interest schema score reflects self-centered moral
reasoning, the Maintaining norms score signifies a commitment to adhering to
societal norms, and the Post-conventionality score represents advanced moral
reasoning based on ethical principles and values. Individual stage-wise score
are defined as follows:

$score_{\theta}=10\cdot\sum_{i=1}^{4}((5-i)\cdot S_{i,\theta})$ (1)

where $S_{i,\theta}$ is defined as:

$S_{i,\theta}=\begin{cases}1&\text{if the $i^{th}$ ranked statement $\in$
Stage-$\theta$}\\\ 0&\text{otherwise}\end{cases}$

Therefore, Personal Interest schema score = $score_{2}+score_{3}$, Maintaining
norms score = $score_{4}$ and $p_{score}=score_{5}+score_{6}$. We present a
working example of the score calculation in the Appendix Section A.1.

##  4 Results and Observation

(a) ChatGPT

(b) Llama2Chat-70B

(c) GPT-4

Figure 1: Dilemma-specific resolution heatmaps across various languages for
ChatGPT, Llama2chat-70B, and GPT-4. O1 is indicated in green, O2 in blue, and
O3 in red. The heatmaps illustrate the number of instances where the models
provided answers corresponding to O1, O2, or O3 for each language and dilemma
based on the RGB component. White areas represent scenarios where no
observations yielded an extractable resolution to the dilemma.

(a) ChatGPT

(b) Llama2Chat-70B

(c) GPT-4

Figure 2: Overview of stage-wise scores for ChatGPT, Llama2Chat, and GPT-4,
averaged across all moral dilemmas. The cumulative scores of the initial three
tiers (Red, Orange, and Deep Yellow) is the $p_{score}$ or post-conventional
morality score. The 4th tier (light yellow) signifies the Maintaining Norms
schema score and the 5th and 6th tiers (green and blue) combined gives the
Personal Interests schema score.

###  4.1 Moral Judgment by the LLMs

As discussed earlier, we consider moral judgment as the resolution of the
dilemmas offered by the LLMs. Figure 1 summarizes the resolutions from
ChatGPT, Llama2chat-70B, and GPT-4 for different languages as a 2-dimensional
color map across languages and dilemmas. Green color signifies the affirmative
side of the resolution, as seen in “should share,” “should take,” and “should
report”, etc. Blue color represents the model’s inability to adopt a specific
moral stance, leading it to select the ”Can’t decide” option. Red color
designates the negative side, exemplified by “should not share,” “should not
take,” and “should not report”, etc. Since the affirmative and negative
statements are rather arbitrary, we list the actual values of these stances
for each dilemma.

  * •

Heinz Should Heinz steal a drug from an inventor in town to save his wife who
is dying and needs the drug?

  * •

Student Should students take over an administration building in protest of the
Vietnam war?

  * •

Newspaper Should a student newspaper be stopped by a Principal of a high
school when the newspaper stirs controversy in the community?

  * •

Webster Should a minority member be hired for a job when the community is
biased?

  * •

Prisoner Should a man who escaped from prison but has since been leading an
exemplary life be reported to authorities?

  * •

Timmy Should Timmy attend his friend’s wedding instead of fixing an urgent bug
that could put customers’ privacy at risk?

  * •

Rajesh Should Rajesh rent a house by hiding the secret of his non-vegetarian
consumption at home from the vegetarian neighborhood?

  * •

Monica Should Monica give the first authorship to Aisha despite having the
major contribution?

  * •

Auroria Should the country Auroria share its innovations and resources to it’s
poor neighbor or profit off it’s huge investments in research?

(a) P-Scores for ChatGPT

(b) P-Scores for Llama2Chat-70B

(c) P-Scores for GPT-4

(d) Maintaining Norms Schema score for ChatGPT

(e) Maintaining Norms Schema score for Llama2Chat-70B

(f) Maintaining Norms Schema Score for GPT-4

(g) Personal Interests Schema score for ChatGPT

(h) Personal Interests Schema score for Llama2Chat-70B

(i) Personal Interests Schema score for GPT-4

Figure 3: Comparing dilemma-specific and overall P-scores among ChatGPT,
Llama2Chat, and GPT-4, versus the random baselines, across five languages for
ChatGPT and Llama2Chat (excluding Hindi) and six languages for GPT-4.

It is evident from the Figure 1 that GPT-4 exhibits a significantly higher
level of consensus in the resolutions across different languages, in
comparison to Llama2Chat and ChatGPT. Quite intriguingly, GPT-4 predominantly
yields ”O3” responses, whereas Llama2Chat tends to produce more ”O1”
responses, and ChatGPT more O2 (“cant’ decide”) responses especially for high-
resource languages like English, Chinese, Russian, and Spanish. It’s worth
noting that all models and languages converge towards an O1 response for the
Webster and Auroria dilemmas. In contrast, for the Student dilemma we observe
a considerable degree of variation in the resolutions across languages for all
models.

Comparing the resolution patterns across languages, we observe that for all
models, resolution in English and Spanish are similar to each other. For
Llama2Chat and GPT-4, moral judgments in Spanish and Chinese are similar,
while those in Russian and English are most different. In contrast, for
ChatGPT, Russian and English resolutions are quite similar, while resolutions
in Swahili and Russian, and in Swahili and Chinese are most dissimilar.
Overall, moral judgments in Russian seem to disagree most with that in other
languages, especially for GPT-4 and Llama2Chat.

It is interesting to speculate the potential reasons behind these differences.
It is possible that for low-resource languages like Hindi and Swahili, the
model does not have exposure to enough pre-training and fine-tuning data to
learn the typical cultural values for the L1 speakers of these languages;
neither the LLMs are capable of performing complex reasoning and processing in
these languages, as has been shown by several recent multilingual benchmarking
studies Ahuja et al. (2023); Wang et al. (2023). Therefore, for these
languages, the resolutions are either random or a direct translation of the
moral resolutions in a high resource language such as English (as if English
was the L1 of the LLM, and languages for which it had very limited
proficiency, such as third language - L3 or fourth language - L4, it
translated the input to English, reasoned over the translated input and
translated the response back to the Language). Indeed, Llama2Chat responded in
English for Swahili and even for Chinese.

On the other hand, for a relatively high resource language, like Spanish,
Chinese and Russian, the LLMs might have had sufficient exposure to data from
which it could learn the cultural values of the L1 speakers of these
languages. According to the World Value Survey, Russia (orthodox European) is
farthest from English speaking countries on the value map (see Fig 4), and
thus, perhaps, elicits the most dissimilar moral judgments compared to
English. On the other hand, Spain (Catholic Europe) is closest (among the
languages we studied) to English on the value map, followed by Chinese and
thus, these languages elicit similar responses to that of English.

Interestingly, the resolutions in Russian and Chinese significantly differ
from each other for all models, despite Russia and China being closely placed
on the value map. A possible explanation for this could be as follows. As Rao
et al. (2023) speculate, the LLMs seem to align to the values on the right-
upper triangle of the map (above the dashed diagonal line in Fig 4). China,
Spain and English speaking countries are on the upper-right triangle, while
Russia falls into the lower-left triangle, which might explain the differences
in the moral judgments. In other words, the behavior of the LLMs seem to
change for languages on the two sides of the dashed line, which could also be
an artifact of the nature of these specific dilemmas.

Figure 4: An illustration of contemporary Language Models with the world
cultural map Rao et al. (2023).

###  4.2 Moral Reasoning by LLMs

As discussed in Section 2.2, moral reasoning is how people think through
what’s right or wrong by using their values and ethical principles. It
involves critical thinking and understanding different ethical ideas, using
both logical and emotional thoughts to make ethical choices Richardson (2003).
In simpler terms, it’s the process behind forming moral judgments. Rest (1986)
shows how moral reasoning can be understood with the help of DIT scores from a
rationalist perspective.

In Figure 2, we can see the stages of cognitive moral development for these
models for different languages. Across all models, CMD tends to be
concentrated in the post-conventional morality stage, with an exception of
ChatGPT for Hindi where its moral reasoning is predominantly centered around
the personal interests schema and Llama2Chat for Swahili, where it is
concentrated around the maintaining norms schema score. For both ChatGPT and
Llama2Chat, there is a more balanced distribution between the two moral
schemas, maintaining norms and personal interest. The average (over all
languages) maintaining norms schema scores of Llama2Chat and ChatGPT are 25.68
and 22.17 respectively, while the average personal interest schema scores are
23.93 and 24.74 respectively. GPT-4 exhibits a notably different pattern. Its
values for these schemas are significantly lower compared to the average post-
conventional schema score (or P-score). For GPT-4. Thus, compared to ChatGPT
and Llama2Chat, GPT-4 has a more developed moral reasoning capability for all
the languages studied. The lowest P-score was observed for Hindi, which too is
greater than 40, and is in the range of P-scores observed in adult humans Rest
(1986).

Figure 3 shows the P-scores, maintaining norms schema scores and personal
interest schema scores for all languages across all dilemmas and models. We
also mark the random baseline score (when the top 4 statements are picked at
random from the 12 moral considerations by a model) for each of these schemas.
We note that for Webster dilemma all models had consensus in moral judgment,
however the moral reasoning for resolving this dilemma lies in the personal
interests schema, indicating rather underdeveloped moral reasoning.
Interestingly, for Heinz dilemma, GPT-4 and ChatGPT exhibit high score in the
personal interest schema for all languages, but Llama2Chat shows high
variation across languages. We further note that the all the models take the
maintaining social norms perspective (Stage 4 specific) while resolving the
Prisoner dilemma with a slight variation across language. In short, even
though, on average we observe post-conventional or near post-conventional
moral reasoning abilities in GPT-4 for all languages, and near post-
conventional moral reasoning for all languages except Swahili for Llama2Chat,
for certain dilemmas the models display conventional or pre-conventional
morality.

Due to paucity of space, we omit several other results. Table 1 in the
Appendix presents a comprehensive report of the P-scores (the most common
single index used in DIT based studies) of the LLMs across all dilemmas and
languages. We also conducted Mann-Whitney U Tests of statistical significance
over various runs. Wherever the P-scores in English are statistically
significantly different ($p<0.05$) from that in another language, the numbers
are shown in bold. The salient observations from this analysis are: (a) For
Webster and Prisoner dilemma, there is no significant difference in P-scores
of the models across languages; (b) GPT-4’s P-scores across languages for
Rajesh and Auroria dilemmas show no significant differences; and (c) for all
models, we observe the maximum statistically significant difference in
P-scores across languages for the Heinz dilemma, followed by the Newspaper
dilemma.

##  5 Discussion and Conclusion

In this first of its kind study of multilingual moral reasoning assessment of
LLMs, we observe that quite unsurprisingly, the moral reasoning capability, as
quantified by the DIT stage scores, of LLMs is highest for English, followed
by Spanish, Russian and Chinese, and lowest for Hindi and Swahili. GPT-4
emerges as the most capable multilingual moral reasoning model with less
pronounced differences in its capabilities in different languages.
Nevertheless, we also observe remarkable variation in moral judgments and
reasoning abilities across dilemmas.

Our work opens up several intriguing questions about LLMs moral reasoning, and
the role of language and cultural values that were presented in form of
textual data during the pre-training, instruction fine-tuning and RLHF stages
of the model. Since these datasets are often unavailable for scrutiny
(especially true for ChatGPT and GPT-4), we can only speculate the reasons for
the differences. It will be interesting to design specific experiments to
probe further into the hypotheses and postulates that have been offered as
plausible explanations in this paper.

## Limitations

This study has some notable limitations. Firstly, the evaluation framework we
used from this work Tanmay et al. (2023) may contain bias, as it include some
dilemmas specifically designed from a Western perspective. Although other
dilemmas also consider diverse cultural viewpoints, the complexity of ethical
perspectives across cultures may not be fully captured. Secondly, our study’s
scope is limited to a few languages, primarily focusing on linguistic
diversity, which may restrict the generalizability of our findings to
languages not included. Additionally, the use of Google Translator for
multilingual dilemma translation carries the potential for translation errors.
Despite these limitations, our research offers insights into cross-cultural
ethical decision-making of LLMs in diverse languages, highlighting the need
for future investigations to address these constraints and strengthen the
robustness of our findings.

## Ethical Concerns

Our results show that GPT-4 is a post-conventional moral reasoner (with scores
comparable to philosophers and graduate students) across most of the languages
studied, and it is at least as good as an average adult human for all
languages on moral reasoning tasks. This might lead people to think that GPT-4
or similar models can be used for making real life ethical decisions. However,
this could be very dangerous as, firstly, our experimental setup is limited to
only 9 dilemmas covering a small set of cultural contexts and values;
secondly, our experiments are limited to 6 languages, which cannot and should
not be generalized to the model’s performance to other languages beyond those
tested. We believe that the current work does not provide sufficient and
reliable ground for using LLMs for making moral judgments.

## References

  * Ahuja et al. (2023) Kabir Ahuja, Rishav Hada, Millicent Ochieng, Prachi Jain, Harshita Diddee, Samuel Maina, Tanuja Ganu, Sameer Segal, Maxamed Axmed, Kalika Bali, et al. 2023.  Mega: Multilingual evaluation of generative ai.  _arXiv preprint arXiv:2303.12528_.
  * Alshomary et al. (2022) Milad Alshomary, Roxanne El Baff, Timon Gurcke, and Henning Wachsmuth. 2022.  The moral debater: A study on the computational generation of morally framed arguments.  _arXiv preprint arXiv:2203.14563_.
  * Araque et al. (2020) Oscar Araque, Lorenzo Gatti, and Kyriaki Kalimeri. 2020.  Moralstrength: Exploiting a moral lexicon and embedding similarity for moral foundations prediction.  _Knowledge-based systems_ , 191:105184.
  * Bang et al. (2023) Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023.  A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity.  _arXiv preprint arXiv:2302.04023_.
  * Bebeau and Brabeck (1987) Muriel J Bebeau and Mary M Brabeck. 1987.  Integrating care and justice issues in professional moral education: A gender perspective.  _Journal of moral education_ , 16(3):189–203.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Čavar and Tytus (2018) Franziska Čavar and Agnieszka Ewa Tytus. 2018.  Moral judgement and foreign language effect: when the foreign language becomes the second language.  _Journal of Multilingual and Multicultural Development_ , 39(1):17–28.
  * Chan et al. (2016) Yuen-Lai Chan, Xuan Gu, Jacky Chi-Kit Ng, and Chi-Shing Tse. 2016.  Effects of dilemma type, language, and emotion arousal on utilitarian vs deontological choice to moral dilemmas in Chinese–English bilinguals.  _Asian Journal of Social Psychology_ , 19(1):55–65.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_.
  * Circi et al. (2021) Riccardo Circi, Daniele Gatti, Vincenzo Russo, and Tomaso Vecchi. 2021.  The foreign language effect on decision-making: A meta-analysis.  _Psychonomic Bulletin & Review_, pages 1–11.
  * Corey et al. (2017) Joanna D Corey, Sayuri Hayakawa, Alice Foucart, Melina Aparici, Juan Botella, Albert Costa, and Boaz Keysar. 2017.  Our moral choices are foreign to us.  _Journal of experimental psychology: Learning, Memory, and Cognition_ , 43(7):1109.
  * Costa et al. (2014) Albert Costa, Alice Foucart, Sayuri Hayakawa, Melina Aparici, Jose Apesteguia, Joy Heafner, and Boaz Keysar. 2014.  Your morals depend on language.  _PLOS ONE_ , 9(4):1–7.
  * Dien (1982) Dora Shu-Fang Dien. 1982.  A Chinese perspective on Kohlberg’s theory of moral development.  _Developmental Review_ , 2(4):331–341.
  * Gert and Gert (2002) Bernard Gert and Joshua Gert. 2002.  The definition of morality.
  * Graham et al. (2013) Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P Wojcik, and Peter H Ditto. 2013.  Moral foundations theory: The pragmatic validity of moral pluralism.  In _Advances in experimental social psychology_ , volume 47, pages 55–130. Elsevier.
  * Greene and Haidt (2002) Joshua Greene and Jonathan Haidt. 2002.  How (and where) does moral judgment work?  _Trends in cognitive sciences_ , 6(12):517–523.
  * Haidt (2001) Jonathan Haidt. 2001.  The emotional dog and its rational tail: a social intuitionist approach to moral judgment.  _Psychological review_ , 108(4):814.
  * Hayakawa et al. (2017) Sayuri Hayakawa, David Tannenbaum, Albert Costa, Joanna D Corey, and Boaz Keysar. 2017.  Thinking more or feeling less? Explaining the foreign-language effect on moral judgment.  _Psychological science_ , 28(10):1387–1397.
  * Hendrycks et al. (2023) Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2023.  Aligning AI With Shared Human Values.
  * Hendy et al. (2023) Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023.  How good are GPT models at machine translation? a comprehensive evaluation.  _arXiv preprint arXiv:2302.09210_.
  * Hoover et al. (2020) Joe Hoover, Gwenyth Portillo-Wightman, Leigh Yeh, Shreya Havaldar, Aida Mostafazadeh Davani, Ying Lin, Brendan Kennedy, Mohammad Atari, Zahra Kamel, Madelyn Mendlen, et al. 2020.  Moral foundations twitter corpus: A collection of 35k tweets annotated for moral sentiment.  _Social Psychological and Personality Science_ , 11(8):1057–1071.
  * Huang et al. (2023) Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023.  Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting.  _arXiv preprint arXiv:2305.07004_.
  * Inglehart and Welzel (2010) Ronald Inglehart and Chris Welzel. 2010.  The WVS cultural map of the world.  _World Values Survey_.
  * Jiang et al. (2021) Liwei Jiang, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, et al. 2021.  Can machines learn morality? The Delphi Experiment.  _arXiv preprint arXiv:2110.07574_.
  * Jiao et al. (2023) Wenxiang Jiao, Wenxuan Wang, JT Huang, Xing Wang, and ZP Tu. 2023.  Is ChatGPT a good translator? Yes with GPT-4 as the engine.  _arXiv preprint arXiv:2301.08745_.
  * Kohlberg (1958) Lawrence Kohlberg. 1958.  _The development of modes of moral thinking and choice in the years 10 to 16_.  Ph.D. thesis, The University of Chicago.
  * Kohlberg (1981) Lawrence Kohlberg. 1981.  The philosophy of moral development: Essays on moral development.  _San Francisco_.
  * Kovač et al. (2023) Grgur Kovač, Masataka Sawayama, Rémy Portelas, Cédric Colas, Peter Ford Dominey, and Pierre-Yves Oudeyer. 2023.  Large language models as superpositions of cultural perspectives.  _arXiv preprint arXiv:2307.07870_.
  * Nguyen et al. (2023) Xuan-Phi Nguyen, Sharifah Mahani Aljunied, Shafiq Joty, and Lidong Bing. 2023.  Democratizing llms for low-resource languages by leveraging their english dominant abilities with linguistically-diverse prompts.  _arXiv preprint arXiv:2306.11372_.
  * Olteanu et al. (2019) Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019.  Social data: Biases, methodological pitfalls, and ethical boundaries.  _Frontiers in big data_ , 2:13.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.
  * Piper (Oct 15, 2020) Kelsey Piper. Oct 15, 2020.  The case for taking AI seriously as a threat to humanity.
  * Rao et al. (2023) Abhinav Rao, Aditi Khandelwal, Kumar Tanmay, Utkarsh Agarwal, and Monojit Choudhury. 2023.  Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMs.  _arXiv preprint arXiv:2310.07251_.
  * Rest (1979) J. Rest. 1979.  _Development in Judging Moral Issues_.  University of Minnesota Press, Minneapolis, MN.
  * Rest (1986) J R Rest. 1986.  _DIT Manual : Manual for the Defining Issues Test_.  University of Minnesota Press, Minneapolis, MN.
  * Rest et al. (1999) James R Rest, Stephen J Thoma, Muriel J Bebeau, et al. 1999.  _Postconventional moral thinking: A neo-Kohlbergian approach_.  Psychology Press.
  * Rest et al. (1994) James R Rest et al. 1994.  _Moral development in the professions: Psychology and applied ethics_.  Psychology Press.
  * Richardson (2003) Henry S Richardson. 2003.  Moral reasoning.
  * Schulman et al. (2022) John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Jacob Menick, Jiayi Weng, Juan Felipe Ceron Uribe, Liam Fedus, Luke Metz, Michael Pokorny, Rapha Gontijo Lopes, Shengjia Zhao, Arun Vijayvergiya, Eric Sigler, Adam Perelman, Chelsea Voss, Mike Heaton, Joel Parish, Dave Cummings, Rajeev Nayak, Valerie Balcom, David Schnurr, Tomer Kaftan, Chris Hallacy, Nicholas Turley, Noah Deutsch, Vik Goel, Jonathan Ward, Aris Konstantinidis, Wojciech Zaremba, Long Ouyang, Leonard Bogdonoff, Joshua Gross, David Medina, Sarah Yoo, Teddy Lee, Ryan Lowe, Dan Mossing, Joost Huizinga, Roger Jiang, Carroll Wainwright, Diogo Almeida, Steph Lin, Marvin Zhang, Kai Xiao, Katarina Slama, Steven Bills, Alex Gray, Jan Leike, Jakub Pachocki, Phil Tillet, Shantanu Jain, Greg Brockman, and Nick Ryder. 2022.  ChatGPT: Optimizing Language Models for Dialogue.  _OpenAI_.
  * Schwartz (2012) Shalom H Schwartz. 2012.  An overview of the Schwartz theory of basic values.  _Online readings in Psychology and Culture_ , 2(1):11.
  * Snarey (1985) John R Snarey. 1985.  Cross-cultural universality of social-moral development: a critical review of Kohlbergian research.  _Psychological bulletin_ , 97(2):202.
  * Tanmay et al. (2023) Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, and Monojit Choudhury. 2023.  Probing the Moral Development of Large Language Models through Defining Issues Test.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_.
  * Trager et al. (2022) Jackson Trager, Alireza S Ziabari, Aida Mostafazadeh Davani, Preni Golazizian, Farzan Karimi-Malekabadi, Ali Omrani, Zhihe Li, Brendan Kennedy, Nils Karl Reimer, Melissa Reyes, et al. 2022.  The Moral Foundations Reddit Corpus.  _arXiv preprint arXiv:2208.05545_.
  * Wang et al. (2023) Bin Wang, Zhengyuan Liu, Xin Huang, Fangkai Jiao, Yang Ding, Ai Ti Aw, and Nancy F Chen. 2023.  SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning.  _arXiv preprint arXiv:2309.04766_.
  * Yao et al. (2023) Jing Yao, Xiaoyuan Yi, Xiting Wang, Jindong Wang, and Xing Xie. 2023.  From instructions to intrinsic human values–a survey of alignment goals for big models.  _arXiv preprint arXiv:2308.12014_.
  * Zeng et al. (2022) Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022.  Glm-130b: An open bilingual pre-trained model.  _arXiv preprint arXiv:2210.02414_.
  * Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022.  OPT: Open Pre-trained Transformer Language Models.  _arXiv preprint arXiv:2205.01068_.
  * Zhao et al. (2023) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023.  A survey of large language models.  _arXiv preprint arXiv:2303.18223_.
  * Zhou et al. (2023) Jingyan Zhou, Minda Hu, Junan Li, Xiaoying Zhang, Xixin Wu, Irwin King, and Helen Meng. 2023.  Rethinking machine ethics–can llms perform moral reasoning through the lens of moral theories?  _arXiv preprint arXiv:2308.15399_.
  * Zhu et al. (2023) Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, and Shujian Huang. 2023.  Multilingual machine translation with large language models: Empirical results and analysis.  _arXiv preprint arXiv:2304.04675_.

##  Appendix A Appendix

###  A.1 Score-Calculation

The DIT presents a moral dilemma and 12 statements that correspond to
different stages of Kohlberg’s Cognitive Moral Development. The respondent has
to answer three questions. Question 1 requires the respondent to give the
moral judgment for the dilemma. Question 2 asks the respondent to evaluate the
significance of each statement in making the moral judgment. The respondent
does not know which statement belongs to which stage of CMD. Question 3 asks
the test taker to choose the 4 most important statements (ranked in order of
importance) that influenced the moral judgment.

There are three metrics in the DIT: Post-conventional schema (Stage 5 and 6),
Maintaining Norms schema (Stage 4), and Personal Interests schema (Stage 2 and
3). The test taker has to choose the 4 most important statements out of the
12, and rank them in order of importance. For example, suppose the respondent
chooses statement #11 as the most important statement, #7 as the second most
important, #9 as third most important and #2 as the fourth most important
statement.

The statements belong to different stages of CMD, but the respondent does not
know which stage each statement belongs to. Consider the case where the stages
for a particular dilemma are as follows: #2 (Stage 3), #7 (Stage 4), #9 (Stage
6) and #11 (Stage 5). The scores for each stage are calculated as follows:

Stage 6 score:

$10\times(4\times 0+3\times 0+2\times 1+1\times 0)=20$

Stage 5 score:

$10\times(4\times 1+3\times 0+2\times 0+1\times 0)=40$

Stage 4 score:

$10\times(4\times 0+3\times 1+2\times 0+1\times 0)=30$

Stage 3 score:

$10\times(4\times 0+3\times 0+2\times 0+1\times 1)=10$

Stage 2 score:

$0$

The final scores for each scores are as follows:

Personal Interests schema score:

$score_{2}+score_{3}=0+10=10$

Maintaining Norms schema score:

$score_{4}=30$

Post-conventional schema score:

$P_{score}=score_{5}+score_{6}=40+20=60$

Model |  Lang. | Heinz | Student | Newspaper | Webster | Prisoner | Timmy | Rajesh | Monica | Auroria | P-Score
---|---|---|---|---|---|---|---|---|---|---|---
ChatGPT | en | 45.74 | 55.83 | 53.33 | 22.13 | 20.83 | 71.04 | 61.46 | 45.96 | 56.52 | 48.09
zh | 30.73$\downarrow$32.8 | 56.21$\uparrow$0.7 | 50.00$\downarrow$6.3 | 18.61$\downarrow$15.9 | 40.00$\uparrow$92.0 | 68.24$\downarrow$4.0 | 63.75$\uparrow$3.7 | 32.70$\downarrow$28.8 | 47.14$\downarrow$16.6 | 45.26$\downarrow$5.9
hi | 20.00$\downarrow$56.3 | 44.00$\downarrow$21.2 | 10.00$\downarrow$81.3 | 31.11$\uparrow$40.6 | – | 40.00$\downarrow$43.7 | 20.00$\downarrow$67.5 | 35.56$\downarrow$22.6 | 30.00$\downarrow$46.9 | 25.63$\downarrow$46.7
ru | 34.05$\downarrow$25.6 | 52.14$\downarrow$6.6 | 47.33$\downarrow$11.3 | 25.52$\uparrow$15.3 | 25.00$\uparrow$20.0 | 55.45$\downarrow$21.9 | 42.78$\downarrow$30.4 | 52.97$\uparrow$15.3 | 45.16$\downarrow$20.1 | 42.27$\downarrow$12.1
es | 35.74$\downarrow$21.9 | 68.12$\uparrow$22.0 | 54.47$\uparrow$2.1 | 27.92$\uparrow$26.2 | 23.95$\uparrow$15.0 | 72.61$\uparrow$2.2 | 70.21$\uparrow$14.2 | 54.22$\uparrow$18.0 | 53.33$\downarrow$5.6 | 51.18$\uparrow$6.4
sw | 28.95$\downarrow$36.7 | 49.03$\downarrow$12.2 | 26.21$\downarrow$50.9 | 18.85$\downarrow$14.8 | 27.19$\uparrow$30.5 | 60.40$\downarrow$15.0 | 50.74$\downarrow$17.4 | 41.15$\downarrow$10.5 | 49.60$\downarrow$12.3 | 39.12$\downarrow$18.7
Llama2Chat | en | 46.47 | 52.75 | 47.67 | 28.06 | 17.23 | 67.78 | 68.57 | 60.26 | 51.28 | 48.9
zh | 27.08$\downarrow$41.7 | 48.29$\downarrow$8.5 | 33.04$\downarrow$30.7 | 30.77$\uparrow$9.7 | 18.46$\uparrow$7.1 | 46.67$\downarrow$31.2 | 46.25$\downarrow$32.6 | 37.94$\downarrow$37.0 | 37.69$\downarrow$26.5 | 36.24$\downarrow$25.9
ru | 19.31$\downarrow$58.5 | 54.29$\uparrow$2.9 | 31.25$\downarrow$34.5 | 24.44$\downarrow$12.9 | 16.67$\downarrow$3.3 | 68.15$\uparrow$0.6 | 45.79$\downarrow$33.2 | 61.67$\uparrow$2.3 | 35.00$\downarrow$31.7 | 40.62$\downarrow$16.9
es | 27.42$\downarrow$41.0 | 46.59$\downarrow$11.7 | 47.65$\downarrow$0.1 | 21.28$\downarrow$24.1 | 21.40$\uparrow$24.2 | 61.19$\downarrow$9.7 | 50.32$\downarrow$26.6 | 57.92$\downarrow$3.9 | 32.75$\downarrow$36.1 | 40.72$\downarrow$16.7
sw | 22.56$\downarrow$51.4 | 27.50$\downarrow$47.9 | 14.67$\downarrow$69.2 | 10.77$\downarrow$61.6 | 35.00$\uparrow$103.1 | 38.46$\downarrow$43.3 | 42.08$\downarrow$38.6 | 25.16$\downarrow$58.3 | 56.00$\uparrow$9.2 | 30.25$\downarrow$38.2
GPT-4 | en | 64.0 | 56.52 | 87.14 | 39.75 | 30.65 | 67.78 | 41.22 | 63.81 | 50.29 | 55.68
zh | 34.29$\downarrow$46.4 | 36.36$\downarrow$35.7 | 79.72$\downarrow$8.5 | 44.88$\uparrow$12.9 | 25.33$\downarrow$17.3 | 72.73$\uparrow$7.3 | 41.40$\uparrow$0.4 | 61.30$\downarrow$3.9 | 48.97$\downarrow$2.6 | 49.44$\downarrow$11.2
hi | 27.03$\downarrow$57.8 | 26.67$\downarrow$52.8 | 58.80$\downarrow$32.5 | 32.78$\downarrow$17.5 | 30.62$\downarrow$0.1 | 56.00$\downarrow$17.4 | 42.61$\uparrow$3.4 | 66.59$\uparrow$4.4 | 40.43$\downarrow$19.6 | 42.39$\downarrow$23.9
ru | 37.93$\downarrow$40.7 | 50.00$\downarrow$11.5 | 77.58$\downarrow$11.0 | 40.77$\uparrow$2.6 | 34.75$\uparrow$13.4 | 68.06$\uparrow$0.4 | 50.00$\uparrow$21.3 | 71.85$\uparrow$12.6 | 48.46$\downarrow$3.6 | 53.27$\downarrow$4.3
es | 60.31$\downarrow$5.8 | 47.10$\downarrow$16.7 | 81.54$\downarrow$6.4 | 42.73$\uparrow$7.5 | 17.22$\downarrow$43.8 | 71.67$\uparrow$5.7 | 46.10$\uparrow$11.8 | 70.86$\uparrow$11.0 | 49.53$\downarrow$1.5 | 54.12$\downarrow$2.8
sw | 40.27$\downarrow$37.1 | 37.50$\downarrow$33.7 | 75.00$\downarrow$13.9 | 27.93$\downarrow$29.7 | 34.00$\uparrow$10.9 | 52.7$\downarrow$22.3 | 40.00$\downarrow$3.0 | 68.61$\uparrow$7.5 | 50.87$\uparrow$1.2 | 47.43$\downarrow$14.8
Table 1: Comparison of model performance across various moral dilemmas in
multiple languages, accompanied by the percentage change in P-scores relative
to English as well as overall P-scores. Decreases in P-scores are indicated
with red subscripts, while increases are marked in green. Figure 5: Prompt
structure illustrated for the Monica’s Dilemma in Hindi

###  A.2 Computational Resources

We deployed the Llama2Chat-70B model on 8 V100 GPUs and the total cost of all
the experiments on this model was 400 GPU hours including failed runs. For
experiments with ChatGPT and GPT-4, we used their APIs and hence we are not
aware of the compute used behind these model APIs.


# DATA-DRIVEN REDUCED ORDER MODELING OF ENVIRONMENTAL HYDRODYNAMICS USING DEEP
AUTOENCODERS AND NEURAL ODES

SOURAV DUTTA1† PETER RIVERA-CASILLAS1 ORIE M. CECIL1 MATTHEW W. FARTHING1 EMMA
PERRACCHIONE2 AND MARIO PUTTI3 1 USACE Engineer Research Development Center,
3909 Halls Ferry Rd, Vicksburg MS 39180, USA, and 2 University of Genoa, Via
Dodecaneso 35, 16146 Genova, Italy, and 3 University of Padua, Via Trieste,
63, 35131 Padova, Italy, and † email<EMAIL_ADDRESS>
###### Abstract

Model reduction for fluid flow simulation continues to be of great interest
across a number of scientific and engineering fields. In a previous work [1],
we explored the use of Neural Ordinary Differential Equations (NODE) as a non-
intrusive method for propagating the latent-space dynamics in reduced order
models. Here, we investigate employing deep autoencoders for discovering the
reduced basis representation, the dynamics of which are then approximated by
NODE. The ability of deep autoencoders to represent the latent-space is
compared to the traditional proper orthogonal decomposition (POD) approach,
again in conjunction with NODE for capturing the dynamics. Additionally, we
compare their behavior with two classical non-intrusive methods based on POD
and radial basis function interpolation as well as dynamic mode decomposition.
The test problems we consider include incompressible flow around a cylinder as
well as a real-world application of shallow water hydrodynamics in an
estuarine system. Our findings indicate that deep autoencoders can leverage
nonlinear manifold learning to achieve a highly efficient compression of
spatial information and define a latent-space that appears to be more suitable
for capturing the temporal dynamics through the NODE framework.

###### keywords:

Data-Driven Model Order Reduction, Autoencoder, Neural Ordinary Differential
Equations, Proper Orthogonal Decomposition, Dynamic Mode Decomposition, Radial
Basis Function Interpolation

S. Dutta, P. Rivera-Casillas, O. M. Cecil, M. W. Farthing, E. Perracchione,
and M. Putti

## INTRODUCTION

The computational challenges faced during high-fidelity numerical simulations
of engineering systems governed by nonlinear partial differential equations
(PDEs), especially in applications involving control [2], optimal design and
multi-fidelity optimization [3], can often be mitigated by the development of
reduced order models (ROMs)[4].

Proper orthogonal decomposition (POD) [5, 6] is a well known method for
extracting a solution-dependent reduced basis space from a set of well-
resolved, high-fidelity snapshots, and is most effective when the coherent
structures of the dataset can be ranked in terms of their energy content. The
POD method has been successfully applied in statistics [7], signal analysis
and pattern recognition [8], ocean models [9], air pollution models [10],
convective Boussinesq flows [11], and Shallow Water Equation (SWE) models [12,
13]. Alternatively, nonlinear dimension reduction techniques such as kernel
POD [14] or deep learning-based approaches like autoencoders [15, 16] have
also been used for extracting a reduced basis. Combining autoencoder-generated
bases with various specialized machine learning algorithms for time series
modeling result in fully non-intrusive reduced order models [17, 18, 19].
Hybrid methods [20, 21] can also be obtained by combining a nonlinear manifold
learning technique like autoencoder for discovering the latent space with an
intrusive method for the temporal dynamics.

Following the identification of the latent space, a reduced representation of
the dynamical system is obtained by a Galerkin or Petrov-Galerkin projection
on to the latent space [13, 22], which typically involves intrusive
modifications of the high-fidelity system operators. This work focuses on non-
intrusive reduced order models (NIROMs) that do not require any knowledge of
the high-fidelity simulator. In such a framework, the evolution of the
expansion coefficients in the latent space is usually computed by the
application of different regression-based methods directly on the high-
fidelity data. These include artificial neural networks (ANNs), in particular
multi-layer perceptrons [23], Gaussian process regression (GPR) [24], and
radial basis function (RBF) [25] interpolation. RBF interpolation in
particular has been shown to be quite successful for nonlinear, time-dependent
partial differential equations (PDEs) [26], nonlinear, parametrized PDEs [25],
and aerodynamic shape optimization [27].

Alternatively, in deep neural networks (DNN) such as ResNet, the evolution of
features over the depth of the network is equivalent to solving an ordinary
differential equation (ODE) of the form $\frac{dz}{dt}=F(z,\theta)$ with the
forward Euler method [28]. With this connection in mind, [29] proposed a
’continuous-depth’ neural network called ODE-Net that effectively replaced the
layers in ResNet-like architectures with a trainable ODE solver. This neural
ordinary differential equation approach (NODE) was further improved in [30,
31] and [32] proposed a NODE generative model that can be efficiently trained
on large-scale datasets. Some applications of the NODE framework include
latent space closure modeling [33], ODE/PDE model identification [34],
modeling of irregularly spaced time series data [35], and modeling of spatio-
temporal information in video signals [36].

Dynamic mode decomposition (DMD) is yet another method for obtaining a reduced
order model. DMD represents the temporal dynamics of a complex, nonlinear
system [37, 38] as the combination of a few linearly evolving, spatially
coherent modes that oscillate at a fixed frequency, and which are closely
related to the eigenvectors of the infinite-dimensional Koopman operator [39].
Several variants of the DMD algorithm have been proposed [2, 40, 41, 42] and
have been successfully applied as efficient ROM techniques for determining the
optimal global basis modes for nonlinear, time-dependent problems [43, 44].
For non-parametrized PDEs, DMD presents an efficient framework that combines
all the three stages of ROM development to learn a linear operator in an
optimal least square sense. However, this approach cannot be directly applied
to parametrized problems [45].

In [1], we explored propagating the dynamics of a latent space formed from POD
modes with neural ODEs. The present work investigates substituting the latent-
space described by POD modes with one learned by an autoencoder. Our results
for the combined autoencoder-NODE approach are compared to other methods like
- a) dimension reduction by POD modes with latent space temporal dynamics
captured by Neural ODEs (POD-NODE), b) dimension reduction via POD and
temporal evolution of the latent space with Radial Basis Functions (POD-RBF),
and c) Dynamic Mode Decomposition (DMD), which serve as benchmarks in our
numerical experiments. The performance of each approach will be evaluated on
sample problems based on incompressible flow around a cylinder and shallow
water hydrodynamics in the context of fast replay applications for complex
fluid-dynamics problems.

## METHODOLOGY

The standard ROM development framework can be divided into three stages:

  1. 1.

identification of a low-dimensional latent (or reduced-order) space,

  2. 2.

representation of the nonlinear dynamical system in terms of the reduced basis
and modeling the evolution of the system of modal coefficients, and

  3. 3.

reconstruction in the high-fidelity space for validation and analysis.

### Dimension reduction

In this work, the dominant features of the unsteady flow-field have been
extracted using a linear modal decomposition technique, POD, and a nonlinear
manifold learning method that relies on deep, fully-connected, multi-layer
perceptron (MLP) autoencoders that are highly expressive and scalable [46].
POD is a popular technique for dimension reduction of the solution manifold of
a dynamical system by determining a linear reduced space spanned by an
orthogonal basis with an associated energetic hierarchy, and which represents
an optimal approximation of the solution manifold with respect to the
$L^{2}$-norm. Given a matrix of high-fidelity system snapshots
${\boldsymbol{\mathrm{S}}}\in\mathbb{R}^{N\times M}$ and a matrix of
orthonormal POD basis vectors ${\boldsymbol{\mathrm{\theta}}}$, the modal
coefficient matrix
${\boldsymbol{\mathrm{Z}}}={\boldsymbol{\mathrm{\Theta}}}^{T}{\boldsymbol{\mathrm{S}}}$
constitutes our training data for the latent space learning methods. [47]
provides an excellent overview of POD as well as a comparison with other
dimension-reduction techniques.

#### Autoencoders

An autoencoder is a type of feedforward neural network that is designed to
learn the identity mapping,
${\boldsymbol{\mathrm{h}}}:{\boldsymbol{\mathrm{v}}}\mapsto\widetilde{{\boldsymbol{\mathrm{v}}}}$
such that
$\widetilde{{\boldsymbol{\mathrm{v}}}}\approx{\boldsymbol{\mathrm{v}}}$ and
${\boldsymbol{\mathrm{h}}}:\mathbb{R}^{N}\mapsto\mathbb{R}^{N}$. This is
accomplished using a two-part architecture. The first part is called an
encoder, ${\boldsymbol{\mathrm{h}}}_{E}$, defined by
${\boldsymbol{\mathrm{z}}}={\boldsymbol{\mathrm{h}}}_{E}({\boldsymbol{\mathrm{v}}};{\boldsymbol{\mathrm{\theta}}}_{E})$
where ${\boldsymbol{\mathrm{z}}}\in\mathbb{R}^{m}$ ($m\ll N$), which maps a
high-dimensional input vector ${\boldsymbol{\mathrm{v}}}$ to a low-dimensional
latent vector ${\boldsymbol{\mathrm{z}}}$.

Figure 1: The architecture of the autoencoder network

The second part is called a decoder, ${\boldsymbol{\mathrm{h}}}_{D}$, defined
as
$\widetilde{{\boldsymbol{\mathrm{v}}}}={\boldsymbol{\mathrm{h}}}_{D}({\boldsymbol{\mathrm{z}}};{\boldsymbol{\mathrm{\theta}}}_{D})$,
which maps the latent vector ${\boldsymbol{\mathrm{z}}}$ to an approximation
$\widetilde{{\boldsymbol{\mathrm{v}}}}$ of the high-dimensional input vector
${\boldsymbol{\mathrm{v}}}$. The combination of the two parts yields an
autoencoder of the form

$\displaystyle{\boldsymbol{\mathrm{h}}}:{\boldsymbol{\mathrm{v}}}\mapsto{\boldsymbol{\mathrm{h}}}_{D}\circ{\boldsymbol{\mathrm{h}}}_{E}({\boldsymbol{\mathrm{v}}}).$
(1)

This autoencoder is trained by computing the optimal values of the parameters
$({\boldsymbol{\mathrm{\theta}}}_{E}^{*},{\boldsymbol{\mathrm{\theta}}}_{D}^{*})$
that minimize the reconstruction error over all the training data

$\displaystyle{\boldsymbol{\mathrm{\theta}}}_{E}^{*},\,{\boldsymbol{\mathrm{\theta}}}_{D}^{*}=\operatorname*{argmin}_{{\boldsymbol{\mathrm{\theta}}}_{E},{\boldsymbol{\mathrm{\theta}}}_{D}}\mathcal{L}({\boldsymbol{\mathrm{v}}},\widetilde{{\boldsymbol{\mathrm{v}}}}),$
(2)

where
$\mathcal{L}({\boldsymbol{\mathrm{v}}},\widetilde{{\boldsymbol{\mathrm{v}}}})$
is a chosen measure of discrepancy between ${\boldsymbol{\mathrm{v}}}$ and its
approximation $\widetilde{{\boldsymbol{\mathrm{v}}}}$. The restriction
$\dim({{\boldsymbol{\mathrm{z}}}})=m\ll N=\dim({{\boldsymbol{\mathrm{v}}}})$
forces the autoencoder model to learn the salient features of the input data
via compression into a low-dimensional space and then reconstructing the
input, instead of directly learning the identity function. It is worth noting
that with the choice of a linear, single-layer encoder of the form
${\boldsymbol{\mathrm{z}}}=H_{E}{\boldsymbol{\mathrm{v}}}$, and a linear,
single-layer decoder of the form
$\widetilde{{\boldsymbol{\mathrm{v}}}}={\boldsymbol{\mathrm{H}}}_{D}{\boldsymbol{\mathrm{z}}}$,
where ${\boldsymbol{\mathrm{H}}}_{E}\in\mathbb{R}^{m\times N}$,
${\boldsymbol{\mathrm{H}}}_{D}\in\mathbb{R}^{N\times m}$, and a squared
reconstruction error as the loss function
$\mathcal{L}(\widetilde{{\boldsymbol{\mathrm{v}}}},{\boldsymbol{\mathrm{v}}})=\lVert{\boldsymbol{\mathrm{v}}}-\widetilde{{\boldsymbol{\mathrm{v}}}}\rVert_{2}^{2}$,
the autoencoder model has been shown to learn the same subspace as that
spanned by the first $m$ POD modes if
${\boldsymbol{\mathrm{H}}}={\boldsymbol{\mathrm{H}}}_{E}={\boldsymbol{\mathrm{H}}}_{D}$.
However, additional constraints are necessary to ensure that the columns of
${\boldsymbol{\mathrm{H}}}$ form an orthonormal basis and follow an energy-
based hierarchical ordering [48].

In this work, autoencoders are employed to generate separate low-dimensional
latent representations of the pressure (depth) and the velocity snapshot data
obtained from the high-fidelity simulation of the numerical examples. The
encoder and decoder neural networks are constructed using fully-connected MLP
architectures, as depicted in Figure 1. As the high-fidelity simulation data
is usually available on a two-dimensional spatial grid, the data is first
flattened and then fed to the autoencoder model.

### Latent space evolution

In this section, we outline the non-intrusive framework for modeling the
evolution of time-series data in the latent space. The first method employs
RBF interpolation which is a classical, data-driven, kernel-based method for
computing an approximate continuous response surface that aligns with the
given multivariate data. More details about the POD-RBF NIROM framework can be
found in [26]. The second technique called NODE is a neural-network based
method to predict the continuous evolution of a vector
${\boldsymbol{\mathrm{c}}}$ over time, that is designed to preserve memory
effects within the architecture.

#### Neural ordinary differential equations

We assume that the time evolution of the modal coefficients of the high-
fidelity dynamical system in the latent space can be modeled using a (first-
order) ODE,

$\displaystyle\frac{d\,{\mathbf{z}}}{dt}=\mathcal{F}(t,\mathbf{z}(t)),\text{
with }\mathbf{z}(0)=\mathbf{z}^{0},\;\mathbf{z}\in\mathbb{R}^{d},d\geq 1.$ (3)

The goal is to obtain a NN approximation $\widehat{\mathcal{F}}$ of the
dynamics function $\mathcal{F}$ such that
$\frac{d\,{\mathbf{z}}}{dt}\approx\text{net}(t,\mathbf{z})=\widehat{\mathcal{F}}(t,\mathbf{z},\boldsymbol{\omega})$.
The full procedure can be outlined as follows:

  1. 1.

Compute the time series of modal coefficients
$[\mathbf{z}^{0},\ldots,\mathbf{z}^{M-1}]$ for $t\in\\{0,\ldots,M-1\\}$ where
$\mathbf{z}^{k}\in\mathbb{R}^{m}$.

  2. 2.

Initialize a NN approximation for the dynamics function
$\widehat{\mathcal{F}}(t,\mathbf{z},\boldsymbol{\omega})$ where
$\boldsymbol{\omega}$ represents the initial NN parameters.

  3. 3.

The NN parameters are optimized iteratively through the following steps.

    1. (a)

Compute the approximate forward time trajectory of the modal coefficients by
solving eq. (3) using a standard ODE integrator as,

$\displaystyle\hat{\mathbf{z}}^{M-1}=ODESolve(\widehat{\mathcal{F}},\boldsymbol{\omega},\mathbf{z}^{0},t^{0},t^{M-1})$
(4)

    2. (b)

The free parameters of the NODE model are
$\\{\boldsymbol{\omega},t^{0},t^{M-1}\\}$. Evaluate the differentiable loss
function
$\mathcal{L}\left(ODESolve(\widehat{\mathcal{F}},\boldsymbol{\omega},\mathbf{z}^{0},t^{0},t^{M-1})-\mathbf{z}^{M-1}\right)$.

    3. (c)

To optimise the loss, compute gradients with respect to the free parameters.
Similar to the usual backpropagation algorithm, this can be achieved by first
computing the gradient $\partial\mathcal{L}/\partial\widehat{\mathbf{z}}(t)$,
and then a performing a reverse traversal through the intermediate states of
the ODE integrator. For a memory-efficient implementation, the adjoint method
[29] can be used to backpropagate the errors by solving an adjoint system for
the augmented state vector
${\boldsymbol{\mathrm{b}}}=[\frac{\partial{\mathcal{L}}}{\partial{\widehat{\mathbf{z}}}},\frac{\partial{\mathcal{L}}}{\partial{\boldsymbol{\omega}}},\frac{\partial{\mathcal{L}}}{\partial{t}}]^{T}$
backwards in time from $t^{M-1}$ to $t^{0}$.

    4. (d)

The gradient
$\frac{\partial{\mathcal{L}}}{\partial{\boldsymbol{\omega}}}(t=0)$ computed in
the previous step is used to update the parameters $\boldsymbol{\omega}$ by
using an optimization algorithm like RMSProp or Adam.

  4. 4.

The trained NODE approximation of the dynamics function can be used to compute
predictions for the time trajectory of the modal coefficients.

Following [1], we adopt the TFDiffEq (https://github.com/titu1994/tfdiffeq)
library that runs on the Tensorflow Eager Execution platform to train the NODE
models. RMSProp is adopted for loss minimization with an initial learning rate
of $0.001$, a staircase decay function with a range of variable decay
schedules, and a momentum coefficient of $0.9$.

As a final point of comparison, we consider the standard Dynamic mode
decomposition (DMD)[37, 38] algorithm, which is a powerful data-driven method
capable of providing an accurate decomposition of a complex system into
spatiotemporal coherent structures that may be used for short-time future-
state prediction.

## NUMERICAL EXPERIMENTS

In this section, we first assess the use of autoencoders for building a
reduced space in which the system dynamics are propagated by NODE for a
benchmark flow problem characterized by the incompressible Navier Stokes
equations (NSE). We compare the performance of this framework with a POD-NODE
method where NODE is employed to capture the evolution of modal coefficients
in a reduced space defined by a POD basis. Moreover, we also examine the
relative performance of different NIROM models for a real-world estuarine flow
application governed by the shallow water equations (SWE). The POD-RBF and DMD
NIROM training runs were performed on a Macbook Pro 2018 with a $2.9$ GHz
6-Core Intel Core i9 processor and 32 GB 2400 MHz DDR4 RAM. The autoencoder
latent-space representations were trained on Vulcanite, a high performance
computer at the U.S. Army Engineer Research and Development Center DoD
Supercomputing Resource Center (ERDC-DSRC). Vulcanite is equipped with NVIDIA
Tesla V100 PCIe GPU accelerator nodes and has 32GB memory/node. Training for
the NODE models was performed in serial on Jim, a high performance computer at
the U.S. Army Engineer Research and Development Center Coastal and Hydraulics
Lab (CHL), which is equipped with 2 Intel Xeon E5-2699 v3 CPUs and 128Gbytes
of memory/node.

### Flow around a cylinder

The problem of two-dimensional, incompressible flow around a cylinder is a
classical benchmark CFD example that simulates a time-periodic fluid flow
through a 2D pipe with a circular obstacle. For further details about the
problem setup please see [1]. High-fidelity simulation data is obtained with
OpenFOAM using an unstructured mesh with $14605$ nodes at $Re=100$, such that
the flow exhibits the periodic shedding of von Karman vortices. $313$ training
snapshots are collected for $t=[2.5,5.0]$ seconds with $\Delta t=0.008$
seconds, and the NIROM predictions are obtained for $t=[2.5,6.0]$ seconds with
$\Delta t=0.002$ seconds.

Id |  Units |  Encoder |  Decoder | Scaling | MSE | Training
---|---|---|---|---|---|---
Range |  ($p,v_{x},v_{y}$) 2-8 |  linear, relu, … |  elu, tanh, … |  |  |
AE1 |  $(5,8,7)$ |  linear |  sigmoid | $[0,1]$ | 2.398e-6 | 9.38 min
AE2 |  $(2,2,2)$ |  linear |  sigmoid | $[0,1]$ | 4.196e-6 | 9.56 min
AE3 |  $(2,2,2)$ |  linear |  tanh | $[-1,1]$ | 2.499e-6 | 9.29 min
AE4 |  $(2,3,3)$ |  linear |  sigmoid | $[0,1]$ | 3.107e-6 | 9.07 min
Table 1: Best Autoencoder architectures for the cylinder example. Models were
trained for $1000$ epochs using the Adam optimizer with initial learning rate
= 1e-3 and momentum = $0.9$.

Several different architectures were explored for the autoencoder model by
varying the model parameters like the dimension of the latent space, the
activation functions, various configurations of the learning rate evolution
during training, and Table 1 shows four of the best architectures for the
cylinder flow example. The second column shows the size of the latent space
for each of the solution variables $p,v_{x},v_{y}$. The third and fourth
columns list the activation functions used in the last hidden encoder and the
last hidden decoder layers, respectively.

Figure 2: Visualization of the modal coefficients of the first two latent
space modes for x-velocity, as generated by POD and the four chosen
autoencoder models for the cylinder example

The fifth column shows the scaling applied to the input data which is directly
determined by the activation function used in the decoder output layer. In
general, it was found that activation functions that required scaled input
data like sigmoid, tanh performed better for the decoder output layer than
some of the (semi-)unbounded ones like linear, relu, and elu. However,
unbounded activation functions like linear and elu were seen to generate a
more accurate and efficient latent space. The encoder and the decoder networks
were made up of four hidden layers with gradually decreasing and gradually
increasing size, respectively, and characterized by the relu activation
function, which were found to generate the least noisy latent representations.
The Adam optimizer is used for training with an initial learning
rate=$10^{-3}$ and momentum=$0.9$. An adaptive learning rate decay algorithm
is employed that monitors the training loss and reduces the learning rate by a
factor of 2 if no improvement is detected for $200$ epochs. The sixth column
of Table 1 lists the total mean square reconstruction error for all three
solution variables, while the last column shows the training times for each
model on two NVIDIA Tesla V100 GPU nodes.

Figure 3: Training characteristics for a fixed autoencoder architecture while
varying the latent space dimension Figure 4: Training characteristics of the
chosen four autoencoder architectures (see Table 1) over 1000 epochs

Figure 2 shows the temporal coefficients for the first two latent modes of
$v_{x}$ using a POD truncation that captures $99\%$ of modal energy content,
and the four chosen AE models. While the POD modal coefficients are arranged
according to the decreasing order of amplitude, this cannot be guaranteed for
the AE-generated spaces. AE2 and AE3 models define a latent space of dimension
2, whereas the AE4 model has 3 latent space units and the AE1 model is
identical to the dimension of the POD bases for each variable - 5 ($p$), 8
($v_{x}$), and 7($v_{y}$). The richer quality of the AE1 latent space leads to
better expressivity which is reflected in the subtler features captured in the
modal coefficients (Figure 2 row 2) and also in the lowest reconstruction
error (Table 1).

Figure 3 shows the evolution of the training loss and the adaptive decay of
the learning rate while training autoencoder models for $v_{x}$ using four
gradually increasing latent space dimensions - $2,3,4,8$. The AE3 architecture
was adopted for these runs. The optimization of the hyperparameters with
respect to the training loss becomes gradually harder as the dimension of the
latent space increases. So the models with smaller latent spaces initially
show a faster reduction in training losses. However, the enhanced expressivity
of the models with higher latent dimensions allow them to reach lower values
of training loss after sufficient epochs of training, whereas the losses for
the models with smaller latent dimensions appear to stagnate. Figure 4 shows
the training loss and the learning rate decay for the four chosen AE models.
The AE1 and the AE4 models with the relatively richer latent spaces achieve
the sharpest reduction in training loss, whereas the latent space dimensions
for models AE2 and AE3 appear to be significant barriers in their training
efficiency.

Id | Lyrs | Units |  Act. |  LR steps, rate | Scaling | Aug. | MSE | Training
---|---|---|---|---|---|---|---|---
Range | 1-4 | 32-512 |  tanh, elu,… |  5k-25k, 0.1-0.9 |  |  |  |
AE1-NODE1 | 1 | 256 |  elu |  10k, 0.3 | No | No | 2.30e-5 | 28.80 hrs
AE1-NODE2 | 1 | 256 |  tanh |  5k, 0.7 | Yes | No | 1.34e-4 | 28.69 hrs
AE1-NODE3 | 1 | 512 |  elu |  5k, 0.5 | No | No | 1.97e-5 | 29.17 hrs
AE1-NODE4 | 1 | 256 |  tanh |  10k, 0.25 | Yes | Yes | 1.49e-4 | 28.27 hrs
AE1-NODE5 | 4 | 64 |  tanh |  5k, 0.5 | Yes | No | 1.33e-4 | 33.08 hrs
Table 2: Results for the best NODE architectures for the cylinder example
using the latent space defined by the AE1 autoencoder model. All NODE models
were trained for 50000 epochs using the fourth-order Runge-Kutta solver and
the RMSProp optimizer with initial learning rate = 1e-3 and momentum = 0.9.

An extensive exploration of the NODE hyperparameter and architecture space for
the cylinder example was reported in [1]. Based on those inferences and some
further numerical study, five architectures were selected, and the results
after training for 50000 epochs using the latent space data generated by the
AE1 model are presented in Table 2. All the models generate accurate
predictions at a finer temporal resolution than the training data, and have
excellent agreement with the high-fidelity solution even while extrapolating
outside the training data ($5\leq t\leq 6$ seconds).

Figure 5: Row1 - Comparison of RMSE of the NODE3 and the NODE5 models using
POD and the AE1 basis; Row2 - Comparison of RMSE of the POD-NODE and the AE-
NODE models with two DMD models with comparable latent space dimensions for
the cylinder example

The first row of Fig. 5 compares the time trajectory of the spatial root mean
square errors (RMSE) in the high-fidelity space for NIROM solutions generated
using the POD and the AE1 basis with the NODE3 and NODE5 configurations. It is
encouraging to note that even though the latent-space sizes are identical
between the POD and the AE1 basis, the AE1-NODE solutions are more accurate,
confirming that the nonlinear AE basis generates a more accurate spatial
compression than a linear POD basis of similar size. The second row of Fig. 5
compares the RMSE between the AE4-NODE3, POD-NODE3, and two DMD NIROM models
with truncation levels of $r=3$ and $r=8$. The AE4-NODE3 model achieves better
accuracy than all the other models even while using the most compressed latent
space ($p,v_{x},v_{y}$:2,3,3). The DMD(8) model is roughly comparable with the
POD-NODE3 model owing to the similarity in their latent space dimensions,
whereas the DMD(3) model fares the worst due to the lack of hierarchical
ordering in its latent basis.

### Shallow water equations

The final numerical example involves flows governed by the depth-averaged SWE
which is written in a conservative residual formulation as

$\boldsymbol{\mathfrak{R}}\equiv\frac{\partial{{\boldsymbol{\mathrm{q}}}}}{\partial{t}}+\frac{\partial{{\boldsymbol{\mathrm{p}}}_{x}}}{\partial{x}}+\frac{\partial{{\boldsymbol{\mathrm{p}}}_{y}}}{\partial{y}}+{\boldsymbol{\mathrm{r}}}=0.$
(5)

Here, the state variable ${\boldsymbol{\mathrm{q}}}=[h,u_{x}h,u_{y}h]^{T}$
consists of the flow depth, $h$, and the discharges in the $x$ and $y$
directions, given by $u_{x}h$ and $u_{y}h$, respectively. Further details
about the flux vectors ${\boldsymbol{\mathrm{p}}}_{x}$,
${\boldsymbol{\mathrm{p}}}_{y}$ and the high-fidelity model equations are
available in [26]. The high-fidelity numerical solutions of the SWE are
obtained using the 2D depth-averaged module of the Adaptive Hydraulics (AdH)
finite element suite, which is a U.S. Army Corps of Engineers (USACE) high-
fidelity, finite element resource for 2D and 3D dynamics [49].

#### Tidal flow in San Diego bay

This numerical example involves the simulation of tide-driven flow in the San
Diego Bay in California, USA.

(a) POD-NODE $u_{x}$ (b) AE-NODE $u_{x}$

(c) POD-NODE error (d) AE-NODE error

Figure 6: NIROM solutions of $u_{x}$ and errors at $t=17.36$ hours for the San
Diego example

The AdH high-fidelity model consists of $N=6311$ nodes, uses tidal data
obtained from NOAA/NOS Co-Ops website at a tailwater elevation inflow boundary
and has no flow boundary conditions everywhere else. Further details are
available in [26].

The training space is generated using $1801$ high-fidelity snapshots obtained
between $t=41$ minutes to $t=50$ hours at a time interval of $\Delta t=100$
seconds. The predicted ROM solutions are computed for the same time interval
with $\Delta t=50$ seconds. A latent space of dimension
$265(p,u_{x},u_{y}:36,115,113)$ is generated by using a POD truncation
tolerance of $\tau_{POD}=5\times 10^{-7}$ for each solution component. The AE2
architecture designed for the cylinder example is modified by including
BatchNormalization layers for each hidden layer, using full batches for
training, and by enforcing a latent space of dimension $30$ for each solution
component. The RBF NIROM approximation is computed using a shape factor,
$c=0.01$. The simulation time points provided as input to the NODE model are
normalized to lie in $t\in[0,1]$. The ‘dopri5’ ODE solver is adopted for
computing the hidden states both forward and backward in time. Learning from
the conclusions of the cylinder example, a network consisting of a single
hidden layer with $256$ neurons is deployed and the RMSProp optimizer with an
initial learning rate of $0.001$, a staircase decay rate of $0.5$ every $5000$
epochs, and a momentum of $0.9$ is utilized for training the model over
$20000$ epochs. For the DMD NIROM, the simulation time points are normalized
to an unit time step, and a truncation level of $r=115$ is used to compute the
DMD eigen-spectrum.

Figure 6 compares the POD-NODE and the AE-NODE NIROM solution fields (top row)
for $u_{x}$ at $t=17.36$ hours and the corresponding error plots (bottom row).
It can be seen that even while using a latent dimension that is three times
smaller than POD, the relative errors for the AE-NODE solution ($0.0279$) is
almost two times lower than that of the POD-NODE solution ($0.0494$). Figure 7
shows the spatial RMSE over time for the depth (left) and the x-velocity
(right) NIROM solutions. The AE-NODE ($30$ modes) solution has comparable
accuracy to the POD-NODE ($36$ modes) and the DMD ($115$ modes) solution for
the depth variable and actually outperforms the POD-NODE ($115$ modes)
solution for the x-velocity variable. Additionally, unlike the RBF NIROM
solution, the AE-NODE solution does not exhibit any appreciable accumulation
of error over time due to the higher-order time-stepping scheme adopted for
NODE.

Figure 7: NIROM RMSEs for the San Diego example

## CONCLUSION

We have studied the combined use of autoencoders as a data-driven method for
identifying an efficient reduced latent space that approximates the solution
manifold of a system of nonlinear, time-dependent PDEs and neural ODEs as a
non-intrusive method for modeling the evolution of the reduced latent space
coefficients for the aforementioned dynamical system. Numerical experiments
were carried out with a benchmark flow problem governed by the incompressible
Navier Stokes equations and a real-world application of estuarine flow
dynamics governed by the two-dimensional shallow water equations. The AE
latent space representation was shown to provide a very high degree of
efficient spatial compression, especially for the advection-dominated shallow
water example. The POD-NODE NIROM formulation demonstrated a stable and
accurate learning trajectory in modeling reduced basis dynamics, even in
comparison to classical ROM techniques utilizing DMD, POD and RBF
interpolation. The AE-NODE formulation also produced encouraging extrapolatory
predictions for the flow around a cylinder example. This presents an exciting
prospect for future exploration as even for an isolated system, unperturbed by
unseen external forcings, truly extrapolative predictions of reduced order
dynamics in flow regimes that do not correspond to the training data is a rare
feature for most well-established ROM frameworks.

This study leads to several promising avenues of research. For instance,
neural architecture search (NAS) tools can be adopted to perform an exhaustive
exploration of the network architecture and model hyperparameter space for a
wide range of flow dynamics in order to gain insight of the learning
trajectory and to design more generalizable AE models with improved
reconstruction accuracy. Moreover, integrating the process of identification
of an AE-based latent space with the modeling of system dynamics using NODE
may lead to significant performance improvements if the two independent
learning problems can be designed to intelligently inform each other. All the
relevant data and codes for this study will be made available in a public
repository at https://github.com/erdc/aenode_nirom upon publication.

## ACKNOWLEDGMENTS

This research was supported in part by an appointment of the first author to
the Postgraduate Research Participation Program at the U.S. Army Engineer
Research and Development Center, Coastal and Hydraulics Laboratory (ERDC-CHL)
administered by the Oak Ridge Institute for Science and Education through an
interagency agreement between the U.S. Department of Energy and ERDC.
Permission was granted by the Chief of Engineers to publish this information.

## References

  * [1] Sourav Dutta, Peter Rivera-Casillas, and Matthew W Farthing.  Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamics.  In Proc. AAAI 2021 Spring Symp. Comb. Artif. Intell. Mach. Learn. with Phys. Sci., Stanford, CA, USA, 2021. CEUR-WS, cs.LG/2104.13962.
  * [2] Joshua L. Proctor, Steven L. Brunton, and J. Nathan Kutz.  Dynamic mode decomposition with control.  SIAM J. Appl. Dyn. Syst., 15(1):142–161, 2016.
  * [3] Benjamin Peherstorfer, Karen Willcox, and M A X Gunzburger.  Optimal model management for multifidelity Monte Carlo estimation.  SIAM J. Sci. Comput., 38(5):A3163–A3194, 2016.
  * [4] P. Benner, S. Gugercin, and K. Willcox.  A survey of projection-based model reduction methods for parametric dynamical systems.  SIAM Rev., 57(4):483–531, 2015.
  * [5] L. Sirovich.  Turbulence and the dynamics of coherent structures. Part I:Coherent structures.  Quart. Appl. Math., 45:561–571, 1987.
  * [6] G. Berkooz, P. Holmes, and J.L. Lumley.  The proper orthogonal decomposition in the analysis of turbulent flows.  Annu. Rev. Fluid Mech., 25(1):539–575, 1993.
  * [7] I.T. Jolliffe.  Principal Component Analysis.  Springer New York, USA, 1986.
  * [8] P. Deheuvels and G.V. Martynov.  A Karhunen-Loeve decomposition of a Gaussian process generated by independent pairs of exponential random variables.  J. Func. Anal., 255(9):2363–2394, 2008.
  * [9] P.T.M. Vermeulen and A.W. Heemink.  Model-reduced variational data assimilation.  Monthly Weather Review, 134:2888–2899, 2006.
  * [10] F. Fang, T. Zhang, D. Pavlidis, C.C. Pain, A.G. Buchanan, and I.M. Navon.  Reduced order modelling of an unstructured mesh air pollution model and application in 2D/3D urban street canyons.  Atmos. Env., 96:96–106, 2014.
  * [11] O. San and J. Borggaard.  Principal interval decomposition framework for POD reduced-order modeling of convective Boussinesq flow.  Int. J. Numer. Methods Fluids, 78(1):37–62, 2015.
  * [12] R. Stefanescu, A. Sandu, and I.M. Navon.  Comparison of POD reduced order strategies for the nonlinear 2D shallow water equations.  Int. J. Numer. Methods Fluids, 76(8):497–521, 2014.
  * [13] A. Lozovskiy, M.W. Farthing, C.E. Kees, and E. Gildin.  POD-based model reduction for stabilized finite element approximations of shallow water flows.  J. Comput. Appl. Math., 302:50–70, 2016.
  * [14] Matteo Salvador, Luca Dede, and Andrea Manzoni.  Non Instrusive Reduced Order Modeling of Parameterized PDEs by Kernel POD and Neural Networks, 2021, math.NA/2103.17152.
  * [15] Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton.  Deep learning for universal linear embeddings of nonlinear dynamics.  Nature Comm., 9(1), 2018.
  * [16] Hojat Ghorbanidehno, Jonghyun Lee, Matthew Farthing, Tyler Hesser, Eric F. Darve, and Peter K. Kitanidis.  Deep learning technique for fast inference of large-scale riverine bathymetry.  Advances in Water Resources, 147:103715, 2021.
  * [17] Francisco J. Gonzalez and Maciej Balajewicz.  Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems, 2018, 1808.01346.
  * [18] Hamidreza Eivazi, Hadi Veisi, Mohammad Hossein Naderi, and Vahid Esfahanian.  Deep neural networks for nonlinear model order reduction of unsteady flows.  Phys. Fluids, 32(10):1–19, 2020.
  * [19] Romit Maulik, Bethany Lusch, and Prasanna Balaprakash.  Reduced-order modeling of advection-dominated systems with recurrent neural networks and convolutional autoencoders.  Phys. Fluids, 33(3), 2021.
  * [20] Kookjin Lee and Kevin T. Carlberg.  Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders.  J. Comput. Phys., 404:108973, 2020, 1812.08373.
  * [21] Youngkyu Kim, Youngsoo Choi, David Widemann, and Tarek Zohdi.  A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder, 2020, math.NA/2009.11990.
  * [22] A. Lozovskiy, M. Farthing, and C. Kees.  Evaluation of Galerkin and Petrov-Galerkin model reduction for finite element approximations of the shallow water equations.  Comput. Methods Appli. Mech. Eng., 318:537–571, 2017.
  * [23] J.S. Hesthaven and S. Ubbiali.  Non-intrusive reduced order modeling of nonlinear problems using neural networks.  J. Comput. Phys., 363:55–78, 2018.
  * [24] M. Guo and J.S. Hesthaven.  Data-driven reduced order modeling for time-dependent problems.  Comput. Methods Appl. Mech. Eng., 345:75–99, 2019.
  * [25] C. Audouze, F. De Vuyst, and P.B. Nair.  Nonintrusive Reduced-Order Modeling of Parametrized Time-Dependent Partial Differential Equations.  Numer. Methods Partial Differ. Equation, 29(5):1587–1628, 2013\.
  * [26] Sourav Dutta, Matthew W. Farthing, Emma Perracchione, Gaurav Savant, and Mario Putti.  A greedy non-intrusive reduced order model for shallow water equations.  J. Comput. Phys., 439:110378, 2021.
  * [27] E. Iuliano and D. Quagliarella.  Aerodynamic shape optimization via non-intrusive POD-based surrogate modelling.  In 2013 IEEE Congr. Evol. Comput. CEC 2013, pages 1467–1474. IEEE, 2013.
  * [28] Lars Ruthotto and Eldad Haber.  Deep Neural Networks Motivated by Partial Differential Equations.  J. Math. Imaging Vis., pages 2–10, 2019.
  * [29] Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud.  Neural Ordinary Differential Equations, 2018, 1806.07366.
  * [30] Amir Gholami, Kurt Keutzer, and George Biros.  Anode: Unconditionally accurate memory-efficient gradients for neural odes, 2019, cs.LG/1902.10298.
  * [31] Emilien Dupont, Arnaud Doucet, and Yee Whye Teh.  Augmented Neural ODEs, 2019, stat.ML/1904.01681.
  * [32] Chris Finlay, Jörn-Henrik Jacobsen, Levon Nurbekyan, and Adam M Oberman.  How to train your neural ODE: the world of Jacobian and kinetic regularization, 2020, stat.ML/2002.02798.
  * [33] Romit Maulik, Arvind Mohan, Bethany Lusch, Sandeep Madireddy, Prasanna Balaprakash, and Daniel Livescu.  Time-series learning of latent-space dynamics for reduced-order model closure.  Phys. D Nonlinear Phenom., 405:132368, 2020.
  * [34] Yifan Sun, Linan Zhang, and Hayden Schaeffer.  NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data.  Proc. Mach. Learn. Res., 107(2016):352–372, 2020.
  * [35] Yulia Rubanova, Ricky T. Q. Chen, and David K Duvenaud.  Latent ordinary differential equations for irregularly-sampled time series.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32, pages 5320–5330. Curran Associates, Inc., 2019.
  * [36] David Kanaa, Vikram Voleti, Samira Kahou, and Christopher Pal.  Simple Video Generation using Neural ODEs.  In Annu. Conf. Neural Inf. Process. Syst. 2019, NeurIPS 2019, Vancouver, BC, Canada, 2019.
  * [37] P.J. Schmid.  Dynamic mode decomposition of numerical and experimental data.  J. Fluid Mech., 656(July 2010):5–28, 2010.
  * [38] J. Nathan Kutz, Steven L. Brunton, Bingni W. Brunton, and Joshua L. Proctor.  Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems.  Society for Industrial and Applied Mathematics, Philadelphia, PA, third edition, 2016.
  * [39] B O Koopman.  Hamiltonian Systems and Transformation in Hilbert Space.  Proc. Natl. Acad. Sci. U. S. A., 17(5):315–8, 1931.
  * [40] J. Nathan Kutz, Xing Fu, and Steven L. Brunton.  Multiresolution dynamic mode decomposition.  SIAM J. Appl. Dyn. Syst., 15(2):713–735, 2016.
  * [41] A.K. Alekseev, D.A. Bistrian, A.E. Bondarev, and I.M. Navon.  On linear and nonlinear aspects of dynamic mode decomposition.  Int. J. Numer. Methods Fluids, 82(6):348–371, 2016.
  * [42] Soledad Le Clainche and Jose M. Vega.  Higher order dynamic mode decomposition.  SIAM J. Appl. Dyn. Syst., 16(2):882–925, 2017.
  * [43] D.A. Bistrian and I.M. Navon.  An improved algorithm for the shallow water equations model reduction: Dynamic Mode Decomposition vs POD.  Int. J. Numer. Methods Fluids, 2015.
  * [44] D.A. Bistrian and I.M. Navon.  Randomized dynamic mode decomposition for nonintrusive reduced order modelling.  Int. J. Numer. Methods Eng., 112:3–25, 2017.
  * [45] Fahad Alsayyari, Zoltán Perkó, Marco Tiberga, Jan Leen Kloosterman, and Danny Lathouwers.  A fully adaptive nonintrusive reduced-order modelling approach for parametrized time-dependent problems.  Comput. Methods Appl. Mech. Eng., 373:113483, 2021.
  * [46] Geoffrey E Hinton and Ruslan R Salakhutdinov.  Reducing the Dimensionality of Data with Neural Networks.  Science, 313:504–507, 2006.
  * [47] Kunihiko Taira, Maziar S. Hemati, Steven L. Brunton, Yiyang Sun, Karthik Duraisamy, Shervin Bagheri, Scott T.M. Dawson, and Chi An Yeh.  Modal analysis of fluid flows: Applications and outlook.  AIAA J., 58(3):998–1022, 2020.
  * [48] Elad Plaut.  From Principal Subspaces to Principal Components with Linear Autoencoders, 2018, stat.ML/1804.10253.
  * [49] C.J. Trahan, G. Savant, R.C. Berger, M. Farthing, T.O. McAlpin, L. Pettey, G.K. Choudhary, and C.N. Dawson.  Formulation and application of the adaptive hydraulics three-dimensional shallow water and transport models.  J. Comput. Phys., 374:47–90, 2018.