# Analyzing the Endeavours of the Supreme Court of India to Transcribe and
Translate Court Arguments in Light of the Proposed EU AI Act

Kshitiz Verma
JK Lakshmipat University, Jaipur.
<EMAIL_ADDRESS>
###### Abstract

The Supreme Court of India has been a pioneer in using ICT in courts through
its e-Courts project in India. Yet another leap, its recent project, _Design,
Development, and Implementation of Artificial Intelligence (AI) solution,
tools for transcribing arguments and Court proceedings at Supreme Court of
India_ , expressed through bid number Ref No. AI Solutions/2023/SCI has
potential to impact the way AI algorithms are designed in India, and not just
for this particular project. In this paper, we evaluate the endeavours of the
Supreme Court of India in light of the state of AI technology as well as the
attempts to regulate AI. We argue that since the project aims to transcribe
and translate the proceedings of the constitutional benches of the Supreme
Court, it has potential to impact _rule of law_ in the country. Hence, we
place this application in High Risk AI as per the provisions to the proposed
EU AI Act. We provide some guidelines on the approach to transcribe and
translate making the maximum use of AI in the Supreme Court of India without
running into the dangers it may pose.

## 1 Introduction

Indian judiciary is burdened by a heavy backlog of cases and it is in a
desperate searchfor innovative methods to address the backlog. The use of
technology has helped Indian judiciary to battle the situation (Verma, 2018).
The technology is growing at a very rapid pace and hence it should leverage
new technologies like artificial intelligence. In a written reply in Rajya
Sabha on 07 April 2022, the then Union Minister of Law and Justice, Shri Kiren
Rijiju had highlighted various use cases of AI in judiciary in India (PIB-
Delhi, 2022). The Supreme Court of India has decided to do a pilot project on
the use of AI for transcribing constitutional court arguments in real time
(SupremeCourtofIndia, 2023). When this paper was written, the Supreme Court
had already asked for participation of technology companies in a bid to use AI
in the court proceedings.

### 1.1 Our Focus on the Supreme Court Bid Document

As the Supreme Court mentions in its bid document that the goal is to leverage
artificial intelligence, machine learning and deep learning for various
purposes including the automated transcription of the proceedings of the the
Supreme Court’s constitutional benches SupremeCourtofIndia (2023). The tasks
that are of interest to us, and are discussed in the paper, are enumerated and
for the sake of completeness, reproduced below.

  1. 1.

Page 40, Section V(b)(i.) _Transcribing of arguments and court proceedings on
real time basis and displaying the same on monitors in the courtroom._

  2. 2.

Page 40, Section V(b)(iii.) _The transcription generated primarily must be in
English language. However, the transcription generated must also be capable of
being translated into the languages stated in the Eight Schedule of the
Constitution of India, 1950._

  3. 3.

Page 40, Section V(b)(iv.) _The AI tool to be developed and deployed must have
an advance level of natural language processing, to understand legal terms,
documents, petitions, judgments, etc. and to automatically classify them in
the relevant specialization._

  4. 4.

Page 40, Section V(b)(v.) _The AI tool to be developed and deployed must have
software and machine learning capabilities, to build a sophisticated hierarchy
of classification models to analyse the contents of documents transcribed
contained in unstructured text, rich text, html, PDF documents, to have a
prediction, intelligent processing, smart classification, content extraction
and summarization._

AI is a constantly evolving scientific area right now. There is no consensus
on any single definition of AI or even threats that it may pose (Hinton,
2023), (Bengio, 2023) (LeCun, 2023), (Ng, 2023). In such times, it may be said
that it is very courageous of the Supreme Court to invite bids for using AI to
improve access to justice in India. However, these endeavours have to be
properly evaluated on legal and technical basis so as to appropriately balance
the need of the judiciary to use AI and the robustness of the technology to be
fit for the purpose.

### 1.2 Elements of High Risk AI in the Supreme Court Project

The early and landmark step of the European Union to frame a law for the use
of artificial intelligence is commendable that has been lauded throughout the
world. The proposed EU AI Act of the European-Parliament (2021) considers that
the AI systems that adversely impact the following may be classified as high
risk AI:

  1. 1.

Health and safety of natural persons

  2. 2.

Adverse impact on fundamental rights of natural persons

  3. 3.

Democracy and rule of law

  4. 4.

Environment

The intention of the Supreme Court is to use AI for constitutional bench
matters, that are binding on courts all over India, including the Supreme
Court benches that have a strength lesser than the said constitutional bench.
Since the jurisdiction of Supreme Court constitutional benches is really
supreme, it may directly impact health and safety of the citizens if a case
with it is about health or any other rights, including fundamental rights. It
may impact the environment if the case is of such nature. Even if all such
impacts may look far fetched, being used in constitutional benches, use of AI
may certainly impact rule of law. In any case, playing the devil’s advocate
and assuming a worst case scenario will only help the Supreme Court to assess
the gravity of application of AI in its proceedings. Such an application of
AI, therefore, may be placed under the high risk category as defined in the
proposed EU AI Act.

Thus, the stakes are high if the Supreme Court wants to utilize the
capabilities of AI for transcribing and translating constitutional benches
proceedings as any mistake made by the AI may really lead to a blunder. For
this reason, borrowing some procedures for conformity assessment from the
proposed EU AI Act may be helpful.

### 1.3 Anil Ambani’s Case

Anil Ambani’s case (EconomicTimes, 2019) is an excellent example of
sensitivity of words used in the Supreme Court judgments and how AI may
catastrophically change meanings. Transcribing of voice may not be a big deal
when a machine learning algorithm on YouTube tries to generate the transcripts
of a random video. However, when it is utilized by the Supreme Court of India,
it no longer remains an ordinary application. It has potential to touch
millions, if not billions of lives in India. We all are aware of the infamous
case in which the word “NOT” was removed from the order and removal of this
one word had resulted in two staff members being terminated from their
services (EconomicTimes, 2019).

#### 1.3.1 Reasonably Foreseeable Misuse

A wrong transcription may have a tremendous impact on rule of law. Human
beings may suffer from automation bias. This may mean that the burden of
correctness may be placed on the technology. Thus, even if the error should
have been caught by human beings, it may creep in due to automation bias.
However, since the transcriptions will be displayed on screens in the court,
someone should be able to figure out such mistakes.

#### 1.3.2 Malicious Manipulation

As above, a wrong transcription may still be acceptable as it may be corrected
in the better versions or by a careful use of the technology. However, if the
technology is weaker in cybersecurity and it is possible to manipulate the AI
models maliciously, then it might be easy to attribute a deliberate malicious
manipulation to wrong transcription by the model. This may lead to more cases
like above and may make the error more difficult to attribute to a human
being.

## 2 Use of AI in Judiciary in Other Jurisdictions

Many researchers in USA have found use cases of applications of AI in judicial
system where too much discretionary power is provided to the courts. For
example in granting bails and asylums. Humans are known to suffer from various
biases and such biases often reflect while granting bail and asylums. In such
cases use of algorithms may be better (Sunstein, 2021) (Jung et al.,
2020)(Kleinberg et al., 2018). Many of these proposals are being considered
for improving access to justice in such scenarios. After a yearlong trial,
Charlotte’s jail population is down almost 20 percent without increasing
crime. At the same time, use of AI in policing has been criticized by many
researchers (Castelvecchi, 2020). In the UK, Durham HART model that was
developed jointly by Durham Constabulary and the University of Cambridge, has
been criticized because of adverse impacts on society and legal perspectives
(Oswald et al., 2018).

Singapore courts have been announcing to use AI in automatic transcription of
court proceedings and a use of case of legal area classification is discussed
in Howe et al. (2019).

We do not focus on providing an exhaustive list here, a more comprehensive
discussion on such related work may be found in Jauhar et al. (2021).

## 3 State of AI for various Applications

AI is not mature yet. It is still evolving and a steady state has not been
achieved so as to say something about its use definitively. One recent example
is degradation in the performance of GPT-4 on some tasks (Chen et al., 2023).
AI’s applications in judiciary may suffer from bias, discrimination, lack of
transparency, loss of autonomy of judicial actors (Leslie et al., 2021).

The intentions of the Supreme Court may be right but the technology is not yet
there where it can be seamlessly used in the Supreme Court for transcribing or
translation. More so when the environment in the court may be noisy or the
goal may be to translate documents to all the languages recognized by the
Constitution of India. Hence, any project carried out in a period of 60 days
is certainly going to fall short of the quality that the hon’ble court is
looking for (SupremeCourtofIndia, 2023). The solution and solution provider
must undergo certain conformity assessments to ensure that the system works
the way it is alleged to be working and all foreseeable risks have been
recognized, acknowledged, mitigated and eliminated. The model should have some
degree of explainability than being a black box like most of the AI models
currently are. Hence, the Supreme Court should create a framework for the
companies solving such complex problems. We are afraid that the current bid
document of the Supreme Court of India is not specific enough to provide
guidelines on the design of the AI to the solution providers.

The Supreme Court can be a torch bearer in the development of explainable AI
in India. If Supreme Court puts forth a requirement of explainability, it
might be an inspiration for many other government agencies to do so, allowing
integration of AI in mainstream governance in a swift manner mitigating the
risks that AI may pose.

We now discuss the state of AI in the two applications that are mentioned in
the Supreme Court bid.

### 3.1 AI in Relation to Transcription of Proceedings

The speech recognition technology is relatively late to take off compared to
vision and natural language processing. One of the revolutionary ideas in
automatic speech recognitions have been implemented by Baevski et al. (2020).
This advance has enabled scaling up to 1,000,000 hours of training data (Zhang
et al., 2022). Even in YouTube, that has got a lot of videos to train on,
accuracy is lesser than desired (Lin, 2022). One of the state of the art
models of speech recognition, developed by OpenAI – Whisper, was trained on
680,000 hours of data (Radford et al., 2023). It still has an average word
error rate (WER) of 12% on various speech datasets. In a more curated Large
Vocabulary Continuous Speech Recognition (LVCSR) dataset the errror rates are
still around 6%. Automatic speech recognition (ASR), like other AI, is also
known to exhibit bias (Martin and Wright, 2022)(Feng et al., 2021). Thus, we
have enough mainstream research literature from automatic speech recognition
to show that the technology is far from being 100% accurate. This calls for
evaluation of the transcription done by any system, particularly, if it is for
judicial use.

Different accents add further complexities to the problem of automatic speech
recognition (Viglino et al., 2019), (Prasad and Jyothi, 2020), (Hinsvark et
al., 2021). The judges in the Supreme Court may come from different states and
may have different accents. Moreover, the accent of Indians is different
compared to those in English speaking nations. This comparison is important
because the most of the success of Automatic Speak Recognition is from western
English datasets and such datasets are baised against the non-native speakers.
Hence, the Supreme Court proceeding is a very particular scenario, much
different from usual transcription scenarios. The datasets required should be
from such distribution only. Hence, it is imperative for the Supreme Court to
provide not just 5 videos but as many as are available so that a good training
dataset may be prepared which may be further used by researchers and companies
alike. AI models are important but so is data (Sambasivan et al., 2021).

### 3.2 AI in Relation to Translating Documents

The success of natural language processing and machine translation has seen
amazing results for English, French, Spanish, Japanese, etc. However, it is
not the same for all languages in the world. Joshi et al. (2020) study the
disparity between languages from the point of view of research conducted and
resources created for processing. This has created a big technological divide
between the languages. They categorize languages in six classes, class 5 being
the maximally resource rich and class 0 being the least. Table 1 provides the
list of the languages provided in the Eighth Schedule of the Constitution of
India along with the class they belong to as per Joshi et al. (2020). As per
classification of languages based on the datasets and labels available, the
success of class 5 languages is less likely to be replicated for other classes
. There is no Indian language in class 5. Hindi and other Indian languages
fall in class 4 or lower. This means that datasets as well as labels for these
languages are several orders of magnitude smaller compared to English. Thus,
the same success stories are not likely to be sung for the translation of
English documents to the languages recognized by the Constitution of India.

Hence, one has to be really careful translating legal texts. Translating legal
documents comes with their own pros and cons. While translations enable a
wider access of the legal provisions, use of various words and their
interpretations in other languages may create confusions too. However, the
pros clearly outweigh cons, it is worth doing it but again some guidelines and
checks have to be put in.

Language | Class
---|---
Assamese | 1
Bengali | 3
Gujarati | 1
Hindi | 4
Kannada | 1
Kashmiri | 1
Konkani | 2
Malayalam | 1
Manipuri | 1
Marathi | 2
Nepali | 1
Oriya | 1
Punjabi | 2
Sanskrit | 2
Sindhi | 1
Tamil | 3
Telugu | 1
Urdu | 3
Bodo | 0
Santhali | 1
Maithili | 1
Dogri | 0
Table 1: Class of 22 languages recognized by the Eighth Schedule of the
Constitution of India as per Joshi et al. (2020). Higher class means higher
availability of data that may enable a high quality processing.

### 3.3 Other Complex Legal Tasks

The bid document mentions to build systems that understand legal terms,
documents, petitions, judgments, etc. and to automatically classify them in
the relevant specialization. The bid document also aims to build a
sophisticated hierarchy of classification models to analyze the contents of
documents transcribed contained in unstructured text, rich text, html, PDF
documents, to have a prediction, intelligent processing, smart classification,
content extraction and summarization.

AI systems have no “understanding” as such. It has even sparked a
philosophical debate on what understanding even means. While classification is
certainly a doable task, preparing training datasets for it is time consuming
and the exact problems that need to be solved also needs to be defined first.
The current text in the bid document is not clear. So, we do not address these
issues in the current paper and focus on transcription and translation only.

## 4 Guidelines for the Implementation Inspired by the Proposed EU AI Act

The stakes are high as constitutional benches are going to use AI in their
proceedings for transcription and translation. Any mistake, if goes unnoticed,
may lead to a blunder. Hence, we need to minimize errors in the output of the
AI system. For this, we need to follow some rules. Our further analysis is
inspired by the proposed EU AI Act (European-Parliament, 2021). Thus, the
application of AI in the Supreme Court Constitutional Bench proceedings may be
classified as a high risk AI that may impact rule of law. For this reason,
some procedures for conformity of high risk AI may be borrowed from the
proposed EU AI Act. The core principles for AI systems as provided in the
proposed EU AI Act are reproduced below:

  1. 1.

Human agency and oversight

  2. 2.

Technical robustness and safety

  3. 3.

Privacy and data governance

  4. 4.

Transparency

  5. 5.

Diversity, non-discrimination and fairness

  6. 6.

Social and environmental well-being

A typical machine learning application has an average cycle of around 6-12
months. Hence, our first suggestion is that instead of making it as a two
months project for a solution provider, it should be much longer project for
the Supreme Court itself. In this paper, by adhering to the principles
enshrined in the proposed EU AI Act that are specific for the requirements of
the Supreme Court, our goal is to provide some guiding principles to the
solution provider that is going to implement the AI project. We enumerate the
required conformity assessment below as provided from Articles 9 to 15 of the
proposed EU AI Act.

### 4.1 Risk Management by the Solution Provider

A risk management system is a continuous iterative process that runs
throughout the entire life cycle of a high-risk AI system. In our context, it
means until the system for automatic transcription of the Supreme Court
arguments is in place, it should be monitored for its output. Once the
application becomes so robust that it looses high risk status, vigilance may
be reduced. The idea is to monitor for identification, estimation and
evaluation of the known and the reasonably foreseeable risks that the high
risk AI system can pose. There is clearly a reasonably foreseeable risk
associated with the applications of AI for transcribing and translation. We
have also seen it before that just omitting one word may lead to catastrophes
and Automatic Speech Recognition systems are known to error on words while
creating transcriptions, i.e., their transcriptions are far from being 100%
accurate. Hence, our suggestions are as follows.

  1. 1.

There must be a continuous monitoring and evaluation of the AI system during
development by the solution provider, as well as for the whole duration it is
in use by the Supreme Court.

  2. 2.

The specific risks that may be posed are identified, estimated and evaluated.
In this case, the system may transcribe words wrongfully, insert or delete
words. Estimation of such errors needs to be done well and the solution must
be evaluated for improving. This may create situations like the Anil Ambani
case. In case of translations, wrong translations may create legal
discrepancies for interpretation which may create more problems than solving
the existing ones.

  3. 3.

Look for effective ways to eliminate, mitigate or reduce the risks identified.
Improve the model and the training data. One of the best ways is to put the
output for the public inspection, which the Supreme Court is already doing by
running the transcriptions on the screens in the court. For translations, it
may upload unreliable copies and ask the relevant legal fraternity to check
for the quality of translation and help improve the quality of the training
dataset by providing better translations.

### 4.2 Data and Data Governance Followed by the Solution Provider

There must be a check on the quality of data being used for training the
transcription and translation AI systems. The Supreme Court must ensure that
the data used for training, validation and testing are appropriate for the
intended purpose of the system. The data used should not exceed what is
required for training the desired system and should not devoid the solution
provider of necessary data. There must be a reasonable attempt to remove all
kinds of biases from the system. The biases may be due to region, accent,
gender or any other basis.

  1. 1.

The foremost thing for the solution provider is to prepare a dataset that is
worthy of solving the problem reasonably well. For the modern deep learning
systems, the bigger the dataset, the better. However, the quality also plays a
crucial role. Transcribing and translation are both supervised learning tasks,
so we need the quality and labelled datasets for both. Given that these tasks
may pose high risk to the rule of law, the solution provider needs to pay
special attention to the quality of the datasets prepared and it should have
the same environment as a typical Supreme Court Constitutional Bench
proceeding has.

  2. 2.

The data should be clearly defined for the purpose at hand, which in this case
is transcribing and translation.

  3. 3.

There is a live streaming of various high courts too. Even that data may be
used for preparing training dataset for transcriptions.

  4. 4.

For translations, the high courts may also be asked to prepare translations of
the judgments in the respective regional language so that the issues of less
data for parallel corpus of English with Indian languages may be addressed.

### 4.3 Transparency of Machine Learning Operations

The said transcription and translation AI systems should be designed and
developed in such a way to ensure that their operations are sufficiently
transparent to enable the solution provider and the Supreme Court and its
registry to reasonably understand the system’s functioning in accordance with
the intended purpose of the AI system.

The degree to which the AI system can provide an explanation for decisions it
takes should also be maximized. In the present case, the explanations for why
some sound was transcribed as a particular word is important. Also, why a
particular translation was chosen for a particular sentence is important.
Thus, relevant information about the Supreme Court actions that may influence
performance, including the type or quality of input data should be included.
Any necessary maintenance and care measures to ensure the proper functioning
of that AI system should also be deployed.

  1. 1.

The Supreme Court is already planning to show the transcriptions so generated
on the screens in the courtroom on the real-time basis. This is a really good
step for enforcing the solution provider towards transparent machine learning
operations, as the AI system’s performance may be seen by the legal fraternity
and any errors may be readily found.

  2. 2.

It may be a even better idea to use colours to interpret the output of the
system. For example, if the system has a high confidence in generating some
transcription, it may colour it in black, output with intermediate confidence
may be coloured yellow and outputs with low confidence may be coloured red. It
may highlight deletions by underscores. It will help the readers to catch
errors effortlessly. However, automation bias may creep in as the text with
high confidence of AI may get less attention from the human readers than it
deserves. A similar colouring scheme may be used for translation too.

  3. 3.

Evaluations should be transparent. The AI system needs to understand if its
performance changes as a function of changes in judge, accent, gender, region,
etc. This should be taken as input in designing the AI systems.

### 4.4 Record keeping by the AI System

The AI system should keep a record of the way it is working by automatic
recording of events (‘logs’). This is to ensure the traceability of the way AI
system is functioning.

  1. 1.

The application should be designed to provide extensive logs for useful and
important events. For example, it is good to keep logs for such instances in
which the systems output is marked as underscore, yellow or red. This enables
to explain better the way the algorithm is behaving.

  2. 2.

In the translation process, each sentence must be assigned a low, intermediate
or high confidence and assigned colours as for the transcription process. The
exact way in which the confidence may be assigned will depend on the
implementation of the AI model.

### 4.5 Human Oversight

As the risk posed by an AI system increases, increased amount of human
oversight is required, proportionate to the risks associated with those
systems. Natural persons in charge of ensuring human oversight need to have
sufficient level of AI literacy and the necessary support and authority to
exercise that function and to allow for thorough investigation, if required.
Human oversight shall aim to minimize the risks to health, safety, fundamental
rights and rule of law. Human oversight shall take into account the specific
risks, the level of automation, and context of the AI system. This is to avoid
automation bias so that we do not consider the output of the system as correct
without questioning or until a damage occurs.

  1. 1.

The Supreme Court should appoint persons with sufficient AI literacy for this
purpose. Their task will be just to evaluate the functioning of the AI sytems.

  2. 2.

The solution provider should explicitly assign the task of human oversight to
its team implementing the solution.

### 4.6 Cyber Security

Since it has impact on the rule of law in India, it is very sensitive from
cybersecurity perspective. Such systems are sensitive to attacks as changing
just one word may lead to catastrophes and cyber criminals may try to change
the transcriptions deliberately by attacking the AI model and its parameters.

  1. 1.

Security should be built in design by default and the security standards
should be equivalent to financial institutions like banks may be put in force.

  2. 2.

There should be a due diligence on making systems secure by keeping a
dedicated team of cyber-security experts from either industry or academia.

  3. 3.

The system should be resilient to an unauthorized change of the model
parameters or any faults or inconsistencies.

  4. 4.

Once a system is compromised, it may be easy to disguise a deliberate act as a
mistake made by the AI model. Hence, the solution should take steps to
minimize malicious manipulation.

### 4.7 Technical Documentation Accompanying the Solution

A technical documentation showing that all the provisions regarding the risk
management, data governance, transparency of machine learning operations,
record keeping, human oversight and cybersecurity are met should be a
mandatory deliverable by the solution provider before putting the AI system in
use by the Supreme Court. It should provide information on all the principles
mentioned above. It should include a general description of the AI system
including:

  1. 1.

The nature of the data likely or intended to be processed by the system.

  2. 2.

The description of the hardware on which the AI system is intended to run.

  3. 3.

A detailed and easily intelligible description of the system’s main
optimization goal or goals.

  4. 4.

A detailed and easily intelligible description of the system’s expected output
and expected output quality.

  5. 5.

A detailed and easily intelligible instructions for interpreting the system’s
output. Report errors in the output of the AI in black/yellow/red
transcriptions mentioned before.

  6. 6.

Some examples of scenarios for which the system should not be used. An
explicit declaration from the solution provider if no such scenario exists.

  7. 7.

A detailed description of the elements of the AI system and the process for
its development, including:

    1. (a)

methods and steps performed for the development of the AI system,

    2. (b)

a description of the architecture, design specifications, algorithms and the
data structures, how they related to one another and provide the overall
processing of the AI system,

    3. (c)

a description of the data obtained, labelling procedures, data cleaning
methodologies, etc.,

    4. (d)

assessment of human oversight, i.e., a detailed report on how human oversight
helped in mitigating the errors that otherwise may creep in,

    5. (e)

validation and testing procedures used, including information about the
validation and testing data used and their main characteristics, metrics used
to measure accuracy, robustness, etc.,

    6. (f)

cybersecurity measures put in place, does deploying this AI exposes the SCI or
its website to cybercriminals in any manner?

  8. 8.

A description of the appropriateness of the performance metrics for the
specific AI system.

  9. 9.

A detailed description of the risk management system.

  10. 10.

A report should be made public for the inspection of academics and civil
society.

## 5 Conclusion

This paper presents the state of the art principles to be applied on AI
systems whose application is in sensitive areas like judiciary. We conclude
that the pilot project taken up by the Supreme Court of India to implement AI
for transcribing the Supreme Court constitutional bench proceedings is
commendable but a bit earlier in time. The success of the project largely
depends on the datasets prepared for the said task. We also discussed the use
of AI to translate legal texts from English to other Indian languages. We
concluded that it is even more difficult task to do such translation. In our
opinion, the Supreme Court of India may take up a long term project for
creation of datasets which may be made available to the public at large. This
will foster research. This will certainly take more than 60 days, as required
by the Supreme Court in the current bid document but it will be a much more
robust solution. Finally, we conclude by suggesting that the company providing
the solution must be asked to provide details and a technical report as
mentioned in Section 4.7 should be submitted.

## References

  * Baevski et al. (2020) Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli.  wav2vec 2.0: A framework for self-supervised learning of speech representations.  _Advances in neural information processing systems_ , 33:12449–12460, 2020.
  * Bengio (2023) Yoshua Bengio.  In montréal, one of the fathers of artificial intelligence warns of an existential threat to mankind.  2023\.  URL https://www.lemonde.fr/en/international/article/2023/06/04/in-montreal-one-of-the-fathers-of-artificial-intelligence-warns-of-an-existential-threat-to-mankind_6029007_4.html.
  * Castelvecchi (2020) Davide Castelvecchi.  Mathematicians urge colleagues to boycott police work in wake of killings.  _Nature_ , 2020.
  * Chen et al. (2023) Lingjiao Chen, Matei Zaharia, and James Zou.  How is chatgpt’s behavior changing over time?, 2023.  arXiv, 2307.09009.
  * EconomicTimes (2019) EconomicTimes.  Supreme court sacks two court staffers over mix-up in Anil Ambani appearance order.  2019\.  URL https://economictimes.indiatimes.com/industry/telecom/telecom-news/supreme-court-sacks-two-court-staffers-over-mix-up-in-anil-ambani-appearance-order/articleshow/68002535.cms.
  * European-Parliament (2021) European-Parliament.  The Proposed EU AI Act.  2021\.  URL https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52021PC0206.
  * Feng et al. (2021) Siyuan Feng, Olya Kudina, Bence Mark Halpern, and Odette Scharenborg.  Quantifying bias in automatic speech recognition.  _arXiv preprint arXiv:2103.15122_ , 2021.
  * Hinsvark et al. (2021) Arthur Hinsvark, Natalie Delworth, Miguel Del Rio, Quinten McNamara, Joshua Dong, Ryan Westerman, Michelle Huang, Joseph Palakapilly, Jennifer Drexler, Ilya Pirkin, et al.  Accented speech recognition: A survey.  _arXiv preprint arXiv:2104.10747_ , 2021.
  * Hinton (2023) Geoffrey Hinton.  Risks of artificial intelligence must be considered as the technology evolves: Geoffrey hinton.  2023\.  URL https://www.utoronto.ca/news/risks-artificial-intelligence-must-be-considered-technology-evolves-geoffrey-hinton.
  * Howe et al. (2019) Jerrold Soh Tsin Howe, Lim How Khang, and Ian Ernst Chai.  Legal area classification: A comparative study of text classifiers on singapore supreme court judgments.  _arXiv preprint arXiv:1904.06470_ , 2019.
  * Jauhar et al. (2021) Ameen Jauhar, Vaidehi Misra, Arghya Sengupta, Partha Chakrabarti, Saptarish Ghosh, and Kripabandhu Ghosh.  Responsible artificial intelligence for the indian justice system.  _Vidhi Legal Policy_ , 2021.  URL https://vidhilegalpolicy.in/wp-content/uploads/2021/04/Responsible-AI-in-the-Indian-Justice-System-A-Strategy-Paper.pdf.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury.  The state and fate of linguistic diversity and inclusion in the NLP world.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020.  doi: 10.18653/v1/2020.acl-main.560.
  * Jung et al. (2020) Jongbin Jung, Connor Concannon, Ravi Shroff, Sharad Goel, and Daniel G Goldstein.  Simple rules to guide expert classifications.  _Journal of the Royal Statistical Society Series A: Statistics in Society_ , 183(3):771–800, 2020.
  * Kleinberg et al. (2018) Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan.  Human decisions and machine predictions.  _The quarterly journal of economics_ , 133(1):237–293, 2018.
  * LeCun (2023) Yann LeCun.  One of the ’godfathers’ of ai says concerns the technology could pose a threat to humanity are ’preposterously ridiculous’.  2023\.  URL https://www.businessinsider.in/tech/news/one-of-the-godfathers-of-ai-says-concerns-the-technology-could-pose-a-threat-to-humanity-are-preposterously-ridiculous/articleshow/101020086.cms.
  * Leslie et al. (2021) David Leslie, Christopher Burr, Mhairi Aitken, Josh Cowls, Michael Katell, and Morgan Briggs.  Artificial intelligence, human rights, democracy, and the rule of law: a primer.  _arXiv preprint arXiv:2104.04147_ , 2021.
  * Lin (2022) Phoebe Lin.  Developing an intelligent tool for computer-assisted formulaic language learning from youtube videos.  _ReCALL_ , 34(2):185–200, 2022.
  * Martin and Wright (2022) Joshua L Martin and Kelly Elizabeth Wright.  Bias in automatic speech recognition: The case of african american language.  _Applied Linguistics_ , page amac066, 2022.
  * Ng (2023) Andrew Ng.  A lone, respected voice disputes ai’s existential threat.  2023\.  URL https://aibusiness.com/responsible-ai/a-lone-respected-voice-disputes-ai-s-existential-threat.
  * Oswald et al. (2018) Marion Oswald, Jamie Grace, Sheena Urwin, and Geoffrey C Barnes.  Algorithmic risk assessment policing models: lessons from the durham hart model and ‘experimental’proportionality.  _Information & communications technology law_, 27(2):223–250, 2018.
  * PIB-Delhi (2022) PIB-Delhi.  Use of advance technology at courts.  2022\.  URL https://pib.gov.in/PressReleaseIframePage.aspx?PRID=1814536.
  * Prasad and Jyothi (2020) Archiki Prasad and Preethi Jyothi.  How accents confound: Probing for accent information in end-to-end speech recognition systems.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3739–3753, 2020.
  * Radford et al. (2023) Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever.  Robust speech recognition via large-scale weak supervision.  In _International Conference on Machine Learning_ , pages 28492–28518. PMLR, 2023.
  * Sambasivan et al. (2021) Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo.  “everyone wants to do the model work, not the data work”: Data cascades in high-stakes ai.  In _proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , pages 1–15, 2021.
  * Sunstein (2021) Cass R Sunstein.  Governing by algorithm? no noise and (potentially) less bias.  _Duke LJ_ , 71:1175, 2021.
  * SupremeCourtofIndia (2023) SupremeCourtofIndia.  Design, development, and implementation of artificial intelligence (ai) solution, tools for transcribing arguments and court proceedings at supreme court of india.  2023\.  URL https://main.sci.gov.in/pdf/TN/23052023_062513.pdf.
  * Verma (2018) Kshitiz Verma.  e-Courts Project: A Giant Leap by Indian Judiciary.  _Journal of Open Access to Law_ , 6(1), 2018.  URL https://ojs.law.cornell.edu/index.php/joal/article/view/77.
  * Viglino et al. (2019) Thibault Viglino, Petr Motlicek, and Milos Cernak.  End-to-end accented speech recognition.  In _Interspeech_ , pages 2140–2144, 2019.
  * Zhang et al. (2022) Yu Zhang, Daniel S Park, Wei Han, James Qin, Anmol Gulati, Joel Shor, Aren Jansen, Yuanzhong Xu, Yanping Huang, Shibo Wang, et al.  Bigssl: Exploring the frontier of large-scale semi-supervised learning for automatic speech recognition.  _IEEE Journal of Selected Topics in Signal Processing_ , 16(6):1519–1532, 2022.