# Demonstrating Rosa: the fairness solution for any Data Analytic pipeline

Kate Wilkinson
illumr Ltd., London, United Kingdom  George Čevora
illumr Ltd., London, United Kingdom
<EMAIL_ADDRESS>
###### Abstract

Most datasets of interest to the analytics industry are impacted by various
forms of human bias. The outcomes of Data Analytics [DA] or Machine Learning
[ML] on such data are therefore prone to replicating the bias. As a result, a
large number of biased decision-making systems based on DA/ML have recently
attracted attention.

In this paper we introduce Rosa, a free, web-based tool to easily de-bias
datasets with respect to a chosen characteristic. Rosa is based on the
principles of Fair Adversarial Networks, developed by illumr Ltd., and can
therefore remove interactive, non-linear, and non-binary bias. Rosa is stand-
alone pre-processing step / API, meaning it can be used easily with any DA/ML
pipeline.

We test the efficacy of Rosa in removing bias from data-driven decision making
systems by performing standard DA tasks on five real-world datasets, selected
for their relevance to current DA problems, and also their high potential for
bias. We use simple ML models to model a characteristic of analytical
interest, and compare the level of bias in the model output both with and
without Rosa as a pre-processing step.

We find that in all cases there is a substantial decrease in bias of the data-
driven decision making systems when the data is pre-processed with Rosa.

A topic that has recently received much attention both in the media and in
academia is the bias in decisions made based on the output of Machine Learning
[ML] algorithms [22][16][10]. This bias is especially problematic when the
result is unfair discrimination between groups on the basis of a protected
characteristic. In UK Law, protected characteristics are defined as age,
disability, gender reassignment, marriage and civil partnership, pregnancy and
maternity, race, religion or belief, sex and sexual orientation. The Equality
Act, 2010 [3], made it illegal to discriminate based on any of these
characteristics.

Discrimination in the output of ML algorithms most often stems from biased
human judgements made in the past. These biased judgements can make up the
dataset which an algorithm is trained on, or they can shape the composition of
the dataset so that it does not represent the true population. If an ML
algorithm is trained to predict biased human judgements, then the bias will be
replicated in decisions made based on the output of the algorithm.

One widely publicised example occurred in 2015, when Amazon revealed that its
experimental recruitment algorithm was heavily biased against women [9]. The
dataset used to train the algorithm comprised successful resumes submitted to
the company over the last 10 years. The technology industry is historically
and currently male-dominated, and therefore the proportion of female resumes
in the dataset was far less than the true proportion of female applicants that
would be suitable for the job today. It is understandable that an algorithm
trained to assess the suitability of applicants would find a link between
successful applicants and gender. If decisions are made based on the
assessments of this algorithm, then an unfair proportion of female applicants
will be hired, propagating the bias further.

Almost every dataset has some degree of bias, and not using biased data at all
is certainly worse than making decisions without any data. This calls for
methods of debiasing data with respect to a given characteristic, so that we
can still use potentially biased datasets without fear of creating algorithms
that discriminate unfairly.

Unfortunately, removing bias from a dataset or algorithm is not
straightforward. Simply removing the column that contains information about
the protected characteristic does not necessarily remove information about
that characteristic from the dataset. Datasets often contain variables which
are proxies for other variables. For example, postcode may be a proxy for
race, or education may be a proxy for gender. As discussed in Čevora 2019 [6],
the most common methods of bias removal do not work particularly well, are
based on subjective notions of fairness and/or are very difficult to
implement.

Fair Adversarial Networks [FANs] are an alternative technique for removing
multivariate, non-binary and/or non-linear bias from a dataset, whilst being
trivial to implement [6]. The FAN methodology encodes a novel, biased dataset
which can subsequently be used with a range of analytical tools, free of bias.
FANs therefore directly tackle the problem of biased datasets in ML.

In December 2019, illumr Ltd. released a tool called Rosa, which uses the
principles of FANs to debias novel datasets using a web-based interface. A
free, demo version of Rosa is available online (at illumr’s website), where
any user can upload a small dataset and debias it with respect to a chosen
characteristic. In particular, Rosa aims to be accessible to data analysts who
may not have much experience with data preparation or advanced Machine
Learning techniques.

In this paper, we demonstrate the effect of Rosa on 5, real-world datasets.
The datasets are selected for their high potential to be used in real-world
decision making, and also their high potential for inherent bias.

To test the effectiveness of Rosa, we perform a simple data analytical task
using each dataset, and examine the degree of bias in the output. We then
repeat the task using datasets which have been processed by Rosa, and
investigate whether the bias has decreased. We find significant/total
reductions in bias on all datasets after processing by Rosa. The models that
are chosen for the analytical tasks are basic, out-of-the box models, in order
to imitate the scenario in which an inexperienced analyst may wish to use
Rosa.

##  1 Data Analytic Examples

###  1.1 Criminal Recidivism

Criminal sanctions in most countries do not only depend solely on the criminal
act that has been committed, but also on personal circumstances (e.g. having
children) and the level of threat the individual poses to society. Of a
particular interest to us is estimation of the likelihood of recidivism for a
criminal defendant, as it is becoming increasingly common to delegate this
task to automated decision-making systems. These systems can have a huge
impact on people’s lives, as individuals deemed unlikely to recidivate may be
released on bail or assigned lower sentences than those deemed otherwise.

Correctional Offender Management Profiling for Alternative Sanctions [COMPAS]
is a technology used by the Department of Justice of the United States [DoJ]
to assess the likelihood of recidivism of criminal defendants and guide bail
decisions. Larson and colleagues [12] found significant racial bias in these
assessments, with African-Americans being assigned significantly higher
recidivism risk scores, independent of whether they recidivated or not. This
means COMPAS is much more likely to wrongly label African-Americans as
potential recidivists than White Americans. Conversely White Americans are
much more likely to be wrongly labelled as non-recidivists.

The DoJ unfortunately defends this pattern of discrimination as being fair
[11]. Their argument relies on the fact that more African-Americans recidivate
according to DoJ statistics. Shockingly, Flores and colleagues, writing in
defense of the DoJ, fail to acknowledge the significant evidence for racial
bias within the criminal justice system itself, with Black populations
suffering from higher rates of stop, search, and arrest, despite research to
suggest that their crime rate is relatively similar to other populations [17,
7, 5, 8, 18, 19]. As a result, crime datasets often imply a much higher rate
of criminality for Black populations than is likely to be the case.

####  1.1.1 Data and Methods

In this section we use a dataset complied by ProPublica that includes the
criminal profiles of 11,757 defendants from Broward County, Florida, along
with an indication of whether they recidivated during a two-year follow-up
study. The dataset is freely available to download at Github. Before starting
the analysis we discarded non-categorical or numerical data, and data relating
to the COMPAS risk-assessment scores allocated to the offenders. We kept data
from only two race categories, _Caucasian_ and _African-American_.

To replicate a recidivism model such as COMPAS we used a basic logistic
regression model from Python’s sklearn package to predict likelihood of
recidivism while excluding the race variable from the data, and looked for
bias in the predictions of our model. We then processed the data with Rosa and
repeated the exact same modelling, checking to see whether the bias had
reduced.

####  1.1.2 Results

As shown in Figures 1 and 2 the distribution of model estimates is clearly
discriminatory against African-Americans when Rosa is not used. This bias
essentially disappeared when the data was pre-processed using Rosa.

|
---|---
|
Figure 1: Without data pre-processing using Rosa (left-hand plots), African-Americans are labelled as having a higher probability of recidivism than their Caucasian counterparts by our model, independent of whether they recidivated or not. Using Rosa as a pre-processing step (right-hand plot), this difference has been almost entirely removed. Note that without considering the estimates for recidivists and non-recidivists separately, any difference in the score distribution for African-Americans and Caucasians could be explained by a different rate of recidivism between the racial categories, as opposed to an unfair pattern of misclassification. | Original data | Rosa pre-processed
---|---|---
| non-recid | recid | non-recid | recid
$\mu$ African-American | 0.37 | 0\. 46 | 0.35 | 0.37
$\mu$ Caucasian | 0.27 | 0.33 | 0.35 | 0.36
$\mu$ Diff | 0.10 | 0.13 | 0.00 | 0.01
$\sigma$ African-American | 0.13 | 0.14 | 0.06 | 0.06
$\sigma$ Caucasian | 0.11 | 0.13 | 0.05 | 0.06
$\sigma$ Average | 0.12 | 0.14 | 0.06 | 0.06
$\mu$ Diff / $\sigma$ Average | 0.85 | 0.92 | 0.00 | 0.17
Figure 2: Caucasian individuals receive significantly lower recidivism
likelihood estimates from our recidivism model when Rosa was not used than
their African-American counterparts, irrespective of whether they later
recidivate or not. The amount of bias in the model has been quantified by
dividing the difference in mean estimate for African-Americans and Caucasians
by the average standard deviation of the two distributions. The higher the
value the greater the bias. Using Rosa has significantly decreased the level
of bias in the estimates both for recidivists and non-recidivists.

The accuracy of the regression model prior to debiasing by Rosa was 67%, and
post-debiasing it was 63%. This drop, while significant is likely due to the
bias inherent in the dataset - which was also used to evaluate accuracy.
Unfortunately the real-world nature of the examples presented in this paper do
not allow unbiased evaluation.

####  1.1.3 Discussion

We have performed a simple DA task to model the chance of recidivism of a
criminal defendant. Similar systems are used across USA to determine a
defendant’s suitability for alternative sanctions and can therefore have a
significant impact on the individual’s life. Without correcting for bias our
model discriminated against African-Americans, who were always considered of
higher recidivism risk whether they recidivated or not. These results are in
line with an analysis of COMPAS - a system developed by Equivant and used by
DoJ that has been demonstrated to perform similar discrimination [12].

Replicating the exact same DA pipeline that resulted in a discriminatory
model, but applying Rosa as a data pre-processing step, resulted in model that
did not discriminate.

It is hardly surprising that the initial model was biased against African-
Americans. The dataset was likely affected by the issues discussed in Section
1.1, such as a heightened rate of stop and search for African-Americans
compared to Caucasians, leading to an over-representation of African-Americans
(and black people in general) in criminal datasets relative to their rate of
criminality.

For example, arrests for marijuana possession are 3.7 times higher for the
black population in the US than the white population, despite similar reported
levels of usage [5]. For the COMPAS dataset, ‘Marijuana’ was one of the most
frequently occurring words in the text describing the charge for each
defendant. Dealing with such systematic bias in criminal datasets is essential
as data-driven decision making becomes more widespread. Here we have
demonstrated that Rosa is a suitable tool for such a task.

###  1.2 Absenteeism at work

DA has an ever-increasing role in Human Resource Management [HR], with
employers aiming to only hire/retain the highest performing employees.
Absenteeism is an important aspect of HR for many organisations and therefore
its estimation is an interesting feature that can be used to screen applicants
during the hiring process. While employers should be considering absenteeism
during the hiring process, it is known to be related to age [14] which is a
protected characteristic. Achieving lower absenteeism in a workforce by age-
skewed hiring would therefore be illegal.

####  1.2.1 Data and Methods

We have performed a simple analysis to model absenteeism in the _Absenteeism
at work_ dataset available from the UCI Machine Learning Repository. This
dataset holds personal information on a set of employees from a courier
company in Brazil, along with their total absenteeism time over three years.

Before commencing with the modelling exercise we removed data on the
type/season of absence, and replaced the ‘Absenteeism Time in Hours’ variable
with a derived feature: ‘Absenteeism in Upper Quartile’, which indicates
whether an employee has absence in the upper quartile of all employee
absences. We also replaced the age (in years) with a categorical variable,
indicating whether employees were under 35, 35 to 45 or over 45. This
simplified the identification of bias as we could compare the model estimates
of absenteeism directly between the three age groups, rather than in a
continuous fashion.

We used the Logistic Regression model from Python’s sklearn package to predict
whether employees had absenteeism in the upper quartile of all the employees
in the dataset while excluding the age variable from the data, and analysed
for bias with respect to age. We then used the same model to predict
absenteeism with data that had been debiased by Rosa, and checked whether the
bias has reduced.

####  1.2.2 Results

|
---|---
|
Figure 3: For both the upper quartile and lower three-quartile absentees, those under 35 received lower estimates of absenteeism than those over 45. The right-hand plots show the model estimates using data pre-processed by Rosa, where most of the difference in model estimates for the over 45 and under 35 groups has been removed. In particular, before Rosa was applied to the dataset, the older lower three-quartile absentees received higher estimates of absenteeism than the younger upper quartile absentees, indicating a high degree of bias with respect to age. This pattern disappeared when Rosa was used to pre-process the data. | Original data | Rosa pre-processed
---|---|---
| upper quart | lower quarts | upper quart | lower quarts
$\mu$ Under 35 | 0.33 | 0.29 | 0.40 | 0.37
$\mu$ Over 45 | 0.52 | 0.37 | 0.41 | 0.35
$\mu$ Diff | 0.19 | 0.08 | 0.01 | 0.02
$\sigma$ Under 35 | 0.05 | 0.10 | 0.17 | 0.15
$\sigma$ Over 45 | 0.16 | 0.14 | 0.11 | 0.08
$\sigma$ Average | 0.11 | 0.12 | 0.12 | 0.12
$\mu$ Diff / $\sigma$ Average | 1.81 | 0.69 | 0.13 | 0.20
Figure 4: Younger individuals receive significantly lower estimates of
absenteeism than their older counterparts when we investigate the upper and
lower three-quartiles of absentees separately in a model without any
compensation of age related bias. The amount of bias in the model has been
quantified by dividing the difference in mean estimate for those over 45 and
those under 35 by the average standard deviation of the two distributions. The
higher the value the greater the bias. Using Rosa has significantly decreased
the level of bias in the estimates both for those with upper quartile
absenteeism and those without.

There was a decrease in prediction accuracy of the model from 72% to 65%,
although the unbiased prediction accuracy cannot be directly compared with the
prediction accuracy of a biased model. This is because predicting certain
outcomes with and without bias are fundamentally different tasks, and
therefore success on these separate tasks cannot be compared directly.

####  1.2.3 Discussion

We demonstrated that a simple model to predict employee absenteeism shows
significant bias with respect to age. This is concerning as it is becoming
increasingly common for automated assessments to form a part of hiring
processes, and predicting absenteeism is of much interest to employers. While
it is acceptable to discriminate based on likely absenteeism alone, we have
seen that this correlates with age in a simple model, which would likely
result in age-based discrimination in hiring, which is illegal.

A particular pattern of discrimination appeared in the analysis of the
original dataset: the older lower three-quartile absentees received higher
estimates of absenteeism than the younger upper-quartile absentees, indicating
a large degree of discrimination with respect to age. This pattern disappeared
when Rosa was used to pre-process the data.

Using the exact same DA pipeline to predict absenteeism after pre-processing
by Rosa, the model estimates showed near zero bias, meaning that the dataset
would be safe to use in an automated hiring procedure without risk of age
discrimination.

###  1.3 Heart Disease

Coronary heart disease [CHD] is the leading cause of death in women [2]. At
certain ages, CHD mortality rates are up to 5x higher in men than women [4].
However, this risk is highly age dependent and means that over a lifetime, men
and women have a relatively similar risk. Despite this, the higher risk for
men during middle-age leads to the common misconception that CHD is a ‘men’s
disease’, in turn leading to a lack of research and data collected on CHD in
women.

Although the major risk factors for CHD in healthy women are the same as those
identified for healthy men in epidemiological studies, the relative strength
of certain risk factors may depend on gender [13][20]. This means that a model
which is trained to predict the risk of heart disease using data mostly from
men may perform less well on women. This can also lead to the misdiagnosis of
heart disease in women, as symptoms that indicate the presence of heart
disease in women are often considered atypical [15].

Misdiagnosis is a serious issue, as the longer a patient goes without
appropriate treatment, the greater the risk of mortality. A study by the
University of Leeds found that women who had a final diagnosis of a STEMI-type
heart attack had a 59 per cent greater chance of a misdiagnosis compared with
men [21].

####  1.3.1 Data and Methods

We used a simple DA model to predict the presence of heart disease from a
dataset of various blood measurements from patients in Cleveland, Ohio. The
_Heart Disease Dataset_ is available at Kaggle. The original dataset has a
scale for type of heart disease, with 0 indicating the absence of heart
disease and 1 to 3 indicating some degree of heart disease. To simplify
modelling, we converted these categories into a binary variable indicating
whether a patient has any degree of heart disease or not.

We used the Logistic Regression model from Python’s sklearn package to predict
the presence of heart disease for individuals in the dataset while excluding
the gender variable from the data, and checked for bias with respect to gender
in the model estimates. We then debiased the data with respect to gender, and
repeated the exact same analytical process to see whether the bias had
decreased.

####  1.3.2 Results

As shown in Figure 5 and 6, the model estimates of heart disease for men are
clearly higher than for women without data pre-processing with Rosa,
regardless of whether the patient had heart disease or not. Using data that
has been pre-processed by Rosa, this bias is significantly reduced.

|
---|---
|
Figure 5: Higher model estimates of heart disease were assigned to men compared to women, both for patients with and without heart disease before the data was pre-processed by Rosa. Using the same model with data pre-processed by Rosa, the model assigns equal estimates to men and women with and without heart disease. | Original data | Rosa pre-processed
---|---|---
| Healthy | Heart Disease | Healthy | Heart Disease
$\mu$ Women | 0.37 | 0.64 | 0.44 | 0.54
$\mu$ Men | 0.44 | 0.67 | 0.44 | 0.54
$\mu$ Diff | 0.07 | 0.03 | 0.00 | 0.00
$\sigma$ Women | 0.24 | 0.18 | 0.11 | 0.12
$\sigma$ Men | 0.20 | 0.16 | 0.11 | 0.11
$\sigma$ Average | 0.22 | 0.16 | 0.11 | 0.11
$\mu$ Diff / $\sigma$ Average | 0.30 | 0.20 | 0.04 | 0.05
Figure 6: Women received lower estimates of heart disease from our regression
model compared to men, independent of whether they had heart disease or not.
This means that women are more likely to be incorrectly given a negative
diagnosis when they actually have heart disease, compared to men. The amount
of bias in the model has been quantified by dividing the difference in mean
estimate for men and women with and without heart disease by the average
standard deviation of the two distributions. The higher the value the greater
the bias. Using Rosa has significantly decreased the level of bias in the
estimates both for those with heart disease and those without.

The accuracy of the model in diagnosing heart disease using biased data was
0.74, and post-Rosa it was 0.67, although it is not reasonable to directly
compare the accuracy of a biased model with the accuracy of an unbiased model.
This is because predicting certain outcomes with and without bias are
fundamentally different tasks.

####  1.3.3 Discussion

We found that a simple model to predict heart disease was biased with respect
to gender. This fits well with research on the under-/mis-diagnosis of heart
disease in women compared to men [21]. The dataset may suffer from two
problems which could lead to biased predictions: 1) the dataset is likely to
contain an over-representation of men compared to the true population of heart
disease sufferers, as women are less likely be diagnosed correctly; and 2)
signals in the data which may lead to good predictions for men do not
necessarily apply to women.

After de-biasing with Rosa, the predictions made using the heart disease
dataset had significantly reduced bias. This has significant implications, as
it means that existing data collected from clinical trials on men can be used
to predict risk of heart disease without disadvantaging women.

###  1.4 Predicting the Economic Needs Index of Schools

As the financial situation of a school can have a large impact on the success
of its students, it is important to identify and direct resources to
financially disadvantaged schools. In New York City, elite schools admit
students based on the results of a single test: the Specialized High Schools
Admissions Test, or SHSAT. Unfortunately, due to the lack of resources
available to schools in deprived areas, and also the link between race and
socio-economic status, there is little diversity in those admitted to these
elite high schools. This is a problem that replicates across many cities and
many countries.

In order to counter this problem, more resources must be directed to under-
performing schools, allowing students from deprived areas to catch-up with
their less deprived counterparts.

A key variable indicating whether students at a particular school are likely
to benefit from additional resources is the economic needs index of the
school. The economic needs index of the school is the average economic needs
index of its pupils, and is based on factors such as whether the student is
eligible for assistance from the state and whether they have lived in
temporary housing [1].

It is important that the economic needs index can be estimated accurately,
however, as race correlates with socio-economic status, there is opportunity
for inadvertent racial discrimination in estimates (as we will demonstrate).

####  1.4.1 Data and Methods

PASSNYC is a not-for-profit organisation that uses public data to identify
promising students in New York’s under-performing schools. They direct
resources to these schools in order to increase the diversity of students
taking SHSAT.

We used a dataset on schools in New York City, compiled by PASSNYC, to predict
the economic needs index of students. The _PASSNYC dataset_ is available at
Kaggle.

We converted the ‘Percent Black / Hispanic’ column to a binary variable which
indicated whether the school had a majority of black students or not, in order
to ease the identification of bias.

We used the Ridge regression model from Python’s sklearn package to predict
the economic needs index of each school while excluding the race variable from
the data, and looked for racial bias in the predictions. We then debiased the
data with respect to race, and made another set of predictions using the
debiased data to see whether the bias had decreased when the data was pre-
processed using Rosa.

####  1.4.2 Results

|
---|---
Figure 7: Before data pre-processing by Rosa, schools with a majority of black students receive much higher estimates of economic needs index than schools with a majority of white students, with very little overlap. Using the same model with data pre-processed by Rosa, the estimates are far less polarised. | True Values | Pre-Rosa | Post-Rosa
---|---|---|---
$\mu$ Majority Black | 0.76 | 0.75 | 0.69
$\mu$ Majority White | 0.42 | 0.44 | 0.57
$\mu$ Diff | 0.34 | 0.31 | 0.12
$\sigma$ Majority Black | 0.13 | 0.08 | 0.11
$\sigma$ Majority White | 0.19 | 0.06 | 0.08
$\sigma$ Average | 0.16 | 0.07 | 0.095
$\mu$ Diff / $\sigma$ Average | 1.93 | 4.43 | 1.26
Figure 8: Prior to data de-biasing using Rosa, the model overestimates the
true difference between the economic needs index of majority black schools and
majority white schools. The amount of bias in the model has been quantified by
dividing the difference in mean estimate for majority black and majority white
schools by the average standard deviation of the two distributions. The higher
the value the greater the bias. Using Rosa has significantly decreased the
level of bias in the estimates both for those with heart disease and those
without. It is clear that the pre-Rosa model overestimates this difference
compared to the true value, and after data pre-processing with Rosa the
difference is far closer to the true value.

The $R^{2}$ for predictions prior to debiasing was 0.57, and post-debiasing it
was 0.35, although we cannot properly assess model accuracy on biased data.

####  1.4.3 Discussion

We found that without using Rosa our model overestimated the economic needs
index of majority black schools, and underestimated the economic needs of
majority white schools. This is problematic as it means that white students
from disadvantaged backgrounds may not receive the same level of support as
similarly disadvantaged black students.

After debiasing the dataset with Rosa, the difference in mean economic needs
index estimate assigned by our model was much closer to the true value than
before de-biasing. This means that organisations like PASSNYC can better
allocate resources by predicting the economic need of students without racial
bias.

###  1.5 Communities and Crime

Being able to predict the rate of crime in different communities is of
interest to law enforcement bodies, as it can allow them to better distribute
resources such as police officers.

However, there is a significant body of research to suggest that Black people
are unfairly represented in criminal datasets due to the racial bias of those
responsible for enacting the law [8, 18, 19]. In certain circumstances, it has
even been found that the true rate of crime for Black persons is likely equal
to that of White persons, despite large differences in the data collected by
law enforcement bodies [5, 17].

This is highly problematic, as any model used to predict crime rate based on
such datasets is likely to overestimate the strength of the true relationship
between race and crime, leading to biased predictions. In this example it
might lead to an unnecessary direction of police resources to communities with
a large Black population. In turn, this is likely to lead to a greater rate of
arrest of Black individuals (as there will be more police officers in areas
with a large Black population), further strengthening the bias in crime
datasets.

####  1.5.1 Data and Methods

We used a dataset on communities and crime within the United States (combining
data from the 1990 US Census, law enforcement data from the 1990 US LEMAS
survey, and crime data from the 1995 FBI UCR) to predict the rate of violent
crime in different communities.

The _Communities and Crime dataset_ is available from the UCI Machine Learning
Repository. We discarded 22 of the most sparsely populated columns, and
converted the ‘RacePercentBlack’ column to a binary label indicating whether
the black proportion in a given community is in the upper quartile across all
communities in the dataset. We removed all other columns that contained
information about the racial profile of each community.

We used the Linear Regression model in Python’s sklearn package to predict the
rate of violent crime in each community while excluding the race variable from
the data, and looked for bias with respect to race. We then debiased the data
with respect to race, and made another set of predictions using the exact same
DA pipeline to see whether the bias had truly been removed.

####  1.5.2 Results

|
---|---
Figure 9: Before the data was pre-processed by Rosa, our model assigns much higher estimates of violent crime rate to majority Black communities compared to majority White communities. After the data has been pre-processed by Rosa, there is less discrepancy between model estimates for majority Black and majority White communities.  | True Values | Pre-Rosa | Post-Rosa
---|---|---|---
$\mu$ Black Upper Quart | 0.49 | 0.51 | 0.39
$\mu$ Black Lower Quarts | 0.16 | 0.15 | 0.20
$\mu$ Diff | 0.33 | 0.36 | 0.19
$\sigma$ Black Upper Quart | 0.27 | 0.17 | 0.29
$\sigma$ White Lower Quarts | 0.16 | 0.13 | 0.17
$\sigma$ Average | 0.215 | 0.15 | 0.23
$\mu$ Diff / $\sigma$ Average | 1.53 | 2.40 | 0.83
Figure 10: There is a large difference in the rate of violent crime between
majority Black and majority White communities in the _Communities and Crime
dataset_ , despite evidence to suggest that the true difference should be
minimal. This is reflected in the estimates from the model before data pre-
processing with Rosa. The amount of bias in the model has been quantified by
dividing the difference in mean estimate for upper-quartile black and lower
three-quartile black communities by the average standard deviation of the two
distributions. The higher the value the greater the bias. The pre-Rosa model
has even greater bias than the original dataset. After data pre-processing
with Rosa, our model estimates have less bias than the original dataset.

The $R^{2}$ for predictions prior to debiasing was 0.69, and post-debiasing it
was 0.52, although we cannot properly assess model accuracy on biased data.

####  1.5.3 Discussion

There is evidence to suggest that there is only a small difference in the rate
of crime for Black persons and White populations, however, most criminal
datasets currently have a large over-representation of Black persons due to
the well documented biases of those enacting the law. This is problematic, as
police resources may be incorrectly distributed based on this data, further
strengthening the existing bias.

We found that our simple model overestimated violent crime rate in communities
with a high proportion of Black residents. After pre-processing the data using
Rosa, there was much less difference in the model estimates for majority Black
and majority White communities.

Although the difference in model estimates for communities with a large
proportion of Black residents and those with a large proportion of White
residents after pre-processing with Rosa was smaller than the true difference
in the dataset, extensive research suggests the dataset itself is biased, and
therefore it cannot be used as a benchmark. We must instead aim for a level of
bias below the true dataset bias in order to prevent the further propagation
of racial bias through the criminal justice system. This was achieved by using
Rosa.

##  2 Discussion

Rosa is a free, web-based tool, which uses the principles of Fair Adversarial
Networks [FANs] to remove bias from a dataset with respect to a certain
characteristic or characteristics. In this paper we demonstrated the bias
removing capabilities of Rosa on five datasets which were related to real and
current issues in the world of Data Analytics [DA] and Machine Learning [ML].
These datasets contained racial, gender or age bias that made the results of
our analysis clearly biased as well; however the bias was successfully
removed/significantly decreased each time we have used Rosa as a pre-
processing step in our analysis.

The main advantage of Rosa is its wide applicability and simplicity for the
end user. Rosa is stand-alone and does not require any integration into the ML
pipeline that a dataset is intended for. It is therefore compatible with a
wide range of ML and data analysis techniques. Less technically minded users
may choose to use a graphical user interface such as the one available at
rosa.illumr.com to pre-process their data for further use in software such as
MS Excel or IBM SPSS. A data scientist comfortable with scripting, on the
other hand, can use the Rosa API directly. In such cases debiasing data
becomes a single line of code.

It should be noted that although we have demonstrated the efficacy of Rosa at
removing bias with respect to protected characteristics [3], Rosa can remove
any type of bias. For example, it might be desirable to remove regional bias
when evaluating performance of an enterprise’s regional offices. In such cases
performance of employees in different parts of the country may not be
comparable without removing the bias of the region.

There were slight decreases in prediction accuracy across all models after
data debiasing. However, because the datasets themselves were all biased in
some respect, we cannot directly compare model accuracy before and after
debiasing. The model trained on the biased dataset was only tested on biased
data - we do not know what its performance would be on fair data. Tests on
synthetic datasets with artificially injected bias indicate that the accuracy
increases when using Rosa (running the DA pipeline with biased data but
evaluating the performance on data before artificial biasing). This result is,
however, reliant on the assumptions about how human biases work and therefore
do not provide a definite proof of increase in accuracy when using Rosa. The
nature of the problem that Rosa addresses makes it impossible to prove an
increase in accuracy in the real-world.

The stochastic nature of both the simple DA models and the FANs behind Rosa
mean that the results of similar experiments will vary every time, and for a
thorough analysis should be repeated multiple times. In our analyses for this
paper, we present a randomly selected run of the DA pipeline to keep the
matter simple. We have observed only very marginal variance in the output with
repeated runs.

As demonstrated in this paper, Rosa is capable of removing many different
types of bias, with no change in approach taken by the end user other than
selecting the right characteristic. This ease-of-use is unrivalled by
alternative bias removal methods.

## References

  * [1] Student economic need index.  Available at: https://data.cccnewyork.org/data/bar/1371/student-economic-need-index#1371/a/1/1622/62, 2019\.
  * [2] II Abubakar, T Tillmann, and A Banerjee.  Global, regional, and national age-sex specific all-cause and cause-specific mortality for 240 causes of death, 1990-2013: a systematic analysis for the global burden of disease study 2013.  Lancet, 385(9963):117–171, 2015.
  * [3] Equality Act.  c. 15.  Retrieved from the UK National Archives website: http://www. legislation. gov. uk/ukpga/2010/15/contents, 2010.
  * [4] Sophie H Bots, Sanne A E Peters, and Mark Woodward.  Sex differences in coronary heart disease and stroke mortality: a global assessment of the effect of ageing between 1980 and 2010.  BMJ Global Health, 2(2), 2017.
  * [5] WC Bunting, Lynda Garcia, and Ezekiel Edwards.  The war on marijuana in black and white.  American Civil Liberties Union, June, 2013.
  * [6] George Cevora.  Fair adversarial networks.  2020\.
  * [7] Dean A Dabney, Laura Dugan, Volkan Topalli, and Richard C Hollinger.  The impact of implicit stereotyping on offender profiling: Unexpected results from an observational study of shoplifting.  Criminal Justice and Behavior, 33(5):646–674, 2006.
  * [8] Dean A Dabney, Richard C Hollinger, and Laura Dugan.  Who actually steals? a study of covertly observed shoplifters.  Justice Quarterly, 21(4):693–728, 2004.
  * [9] Jeffrey Dastin.  Amazon scraps secret ai recruiting tool that showed bias against women.  Reuters, 2018.
  * [10] Virginia Eubanks.  Automating inequality: How high-tech tools profile, police, and punish the poor.  St. Martin’s Press, 2018.
  * [11] Anthony W Flores, Kristin Bechtel, and Christopher T Lowenkamp.  False positives, false negatives, and false analyses: A rejoinder to machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks.  Fed. Probation, 80:38, 2016.
  * [12] Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin.  How we analyzed the compas recidivism algorithm.  ProPublica (5 2016), 9, 2016.
  * [13] JoAnn E Manson, Heather Tosteson, Paul M Ridker, Suzanne Satterfield, Patricia Hebert, Gerald T O’Connor, Julie E Buring, and Charles H Hennekens.  The primary prevention of myocardial infarction.  New England journal of medicine, 326(21):1406–1416, 1992.
  * [14] Joseph J Martocchio.  Age-related differences in employee absenteeism: a meta-analysis.  Psychology and Aging, 4(4):409, 1989.
  * [15] Jean C McSweeney, Leanne L Lefler, and Beth F Crowder.  What’s wrong with me? women’s coronary heart disease diagnostic experiences.  Progress in Cardiovascular Nursing, 20(2):48–57, 2005.
  * [16] Cathy O’Neil.  Weapons of math destruction: How big data increases inequality and threatens democracy.  Broadway Books, 2016.
  * [17] Alex R Piquero and Robert W Brame.  Assessing the race–crime and ethnicity–crime relationship in a sample of serious adolescent delinquents.  Crime & Delinquency, 54(3):390–422, 2008.
  * [18] George E Schreer, Saundra Smith, and Kirsten Thomas.  “shopping while black”: Examining racial discrimination in a retail setting 1.  Journal of Applied Social Psychology, 39(6):1432–1444, 2009.
  * [19] Andy Shallice and Paul Gordon.  Black People, White Justice?: Race and the Criminal Justice System.  Runnymede Trust London, 1990.
  * [20] Eric Vittinghoff, Michael G Shlipak, Paul D Varosy, Curt D Furberg, Christine C Ireland, Steven S Khan, Roger Blumenthal, Elizabeth Barrett-Connor, Stephen Hulley, et al.  Risk factors and secondary prevention in women with heart disease: the heart and estrogen/progestin replacement study.  Annals of internal medicine, 138(2):81–89, 2003.
  * [21] Jianhua Wu, Chris P Gale, Marlous Hall, Tatendashe B Dondo, Elizabeth Metcalfe, Ged Oliver, Phil D Batin, Harry Hemingway, Adam Timmis, and Robert M West.  Editor’s choice-impact of initial hospital diagnosis on mortality for acute myocardial infarction: A national cohort study.  European Heart Journal: Acute Cardiovascular Care, 7(2):139–148, 2018.
  * [22] James Zou and Londa Schiebinger.  Ai can be sexist and racist—it’s time to make it fair, 2018.


# Accelerating Radio Spectrum Regulation
Workflows with Large Language Models (LLMs)

Amir Ghasemi, Paul Guinand  Communications Research Centre (CRC) Canada
Ottawa, Ontario, Canada
Email: {amir.ghasemi<EMAIL_ADDRESS>
###### Abstract

Wireless spectrum regulation is a complex and demanding process due to the
rapid pace of technological progress, increasing demand for spectrum, and a
multitude of stakeholders with potentially conflicting interests, alongside
significant economic implications. To navigate this, regulators must engage
effectively with all parties, keep pace with global technology trends, conduct
technical evaluations, issue licenses in a timely manner, and comply with
various legal and policy frameworks.

In light of these challenges, this paper demonstrates example applications of
Large Language Models (LLMs) to expedite spectrum regulatory processes. We
explore various roles that LLMs can play in this context while identifying
some of the challenges to address. The paper also offers practical case
studies and insights, with appropriate experiments, highlighting the
transformative potential of LLMs in spectrum management.

###### Index Terms:

large language models, artificial intelligence, spectrum management, spectrum
regulation, generative AI

##  I Introduction

Wireless spectrum regulation is complex and challenging given the rapid pace
of technological progress, increasing demand for spectrum, and a wide range of
stakeholders with competing interests. Regulatory processes often involve
labour-intensive tasks such as reviewing technical submissions, coordinating
frequency assignments among various users, consulting with industry
stakeholders, enforcing compliance with regulations, and updating policy
frameworks to reflect the latest technological advancements and market needs.
In this paper we investigate the possible use of Generative Artificial
Intelligence (GenAI) to expedite these processes.

GenAI is a branch of AI that involves generating new outputs based on one or
more inputs. Both inputs and outputs can take various formats such as text,
image, audio, video, or other forms of data. Within this context, text-to-text
models where a text input (prompt) is used to produce a text output have
various applications such as document summarization, question-answering,
language translation, and computer code generation, to name a few. Techniques
developed for these tasks are collectively referred to as natural language
processing (NLP), within which, LLMs form a subset.

Today, transformers [1] form the building blocks of LLMs, which are
distinguished by their size (some containing hundreds of billions of
parameters or more) and their ability to perform a wide range of language
tasks without task-specific training. They can write essays, summarize text,
translate languages, and even generate code, demonstrating a level of
linguistic capability and versatility that was hard to imagine a decade ago.

The remainder of this paper is organized as follows: Key applications of LLMs
in spectrum regulation domain as well as some of their main challenges and
limitations in this context are presented in Sections II and III,
respectively. Instances of LLMs’ practical applications in regulatory tasks
are demonstrated in Section IV, followed by a summary of key practical lessons
and considerations in Section V. Finally, Section VI provides some concluding
remarks.

##  II Impact of LLMs on Radio Spectrum Regulation

This section outlines a number of regulatory tasks that can be improved with
LLMs. We will discuss how LLMs can streamline and accelerate processes, aid in
decision-making, and ensure more holistic responses to regulatory inquiries.

###  II-A Stakeholder Consultations

Preparing radio spectrum regulatory consultations typically involves
conducting an initial research phase as well as having an understanding of
legal frameworks, diverse policy implications, and industry-specific
technicalities. LLMs can expedite the research phase by summarizing relevant
international efforts (e.g., those of ITU or other regional bodies) or
approaches taken by other national regulators within their jurisdictions. They
can also provide a summary of the feedback and comments received from various
types of stakeholders in prior or similar consultations to help regulators
better anticipate and understand potential concerns. LLMs can also help in
automating the drafting process via providing initial templates, suggesting
language based on past successful consultations, and ensuring compliance with
legal and procedural norms. This not only saves time but also reduces the
potential for human error. Furthermore, LLMs can assist with processing and
distilling comments received through the consultation process to speed up the
final decision-making.

###  II-B Rules as Code

Radio spectrum regulations such as radiated power and antenna height limits
could vary based on frequency, geographic location, application, and existence
of multiple licensees. These rules are typically embedded within regulatory
documents and are often wrapped in legal and technical language, making it
difficult for end-users to understand and comply. The Rules as Code (RaC)
concept aims to address this difficulty by converting legislative rules,
regulations, and policies embedded within both structured and unstructured
data sources into machine-readable code [2, 3]. Thus, RaC represents a
significant shift towards integrating technology with legislative processes to
ensure that regulations can be more effectively administered, interpreted, and
applied in a consistent and accessible manner.

LLMs offer a one-stop solution for RaC as they are 1) capable of understanding
complex legal and regulatory documents and databases, 2) can identify and
extract rules, conditions, and constraints that would be applicable under
different scenarios from these sources [4], 3) can directly codify these rules
into various programming languages thanks to their code generation
capabilities, and 4) can generate test cases to validate the code to ensure it
accurately reflects the original regulatory intent.

###  II-C Knowledge-Base Question Answering

One of the major issues faced by spectrum regulators is the time-consuming
task of sifting through many sources of information such as policy and
regulatory documents, technical specifications, and license conditions, in
order to be able to answer regulatory inquiries. LLMs can process and analyze
large amounts of data rapidly, providing quick responses to queries.
Automating the question-answering process with LLMs can reduce the need for
extensive human labor, cutting down on the time and cost associated with
regulatory inquiries.

In addition to saving time and labor, by drawing on authoritative knowledge
bases, LLMs can provide precise and more holistic responses to complex
regulatory queries requiring analysis of multiple factors and data sources.
Furthermore, LLMs make information more accessible to non-experts by
translating technical regulatory language into more understandable terms, thus
enhancing the capability of less experienced staff to handle more complex
problems efficiently. This has the added benefit of democratizing access to
information for the general public.

A practical implementation of such question-answering system will be detailed
in Section IV-B.

###  II-D Automating Processes via LLM Agents

LLMs have the potential to revolutionize radio spectrum regulatory processes
by functioning as autonomous agents capable of executing tasks efficiently and
accurately. While LLMs are not inherently agents, as they do not have any
goals, they can be prompted to act like one [5, 6]. These LLM-powered agents
can be instrumental in streamlining administrative procedures, particularly in
contexts such as license issuance and renewal, coexistence studies, and
interference complaint investigations among other things. This type of
advanced application is enabled by LLM agents’ reasoning capability where they
can break down complex requirements into a series of individual tasks, choose
the right tool for each task (e.g, run a query against a database, search the
Internet for latest industry developments, or run a radio propagation
simulator), and eventually interpret, summarize, and present the final output.

####  II-D1 Training LLMs as Regulatory Agents

LLMs can be trained to act as regulatory agents by leveraging their natural
language understanding and generation capabilities. Through appropriate
training and fine-tuning, these models can develop an understanding of the
intricate rules, regulations, and procedures governing radio spectrum
management. They can process complex legal documents, identify relevant
clauses, and interpret them in the context of specific regulatory tasks, query
various databases, run radio network or coexistence simulations via
application-specific programming interfaces (APIs) and even interpret the
final results to make recommendations. This enables LLMs to act as
knowledgeable agents complementing the human expertise, capable of
comprehending regulatory requirements and making informed choices
autonomously.

####  II-D2 Streamlining Administrative Processes

LLM-powered agents hold significant promise in streamlining administrative
processes, particularly those related to license issuance and renewal. These
tasks often involve extensive paperwork, data verification, and adherence to
regulatory compliance. LLM agents can efficiently handle data entry, document
verification, and even communication with applicants or license holders. By
automating these routine, yet time-consuming tasks, LLMs can reduce the
administrative burden on regulatory bodies, leading to quicker and more
efficient processes. This can result in faster response times, reduced
paperwork, and minimized errors in the regulatory workflow.

####  II-D3 Importance of Human Oversight

While LLM agents offer substantial advantages in terms of efficiency and
accuracy, it is crucial to emphasize the importance of human oversight in LLM-
agent-assisted processes. Regulatory decisions involving spectrum can have
far-reaching consequences and human judgment remains essential in cases
involving ambiguity, novel applications, or fairness considerations. Human
oversight ensures that the decisions made by LLM agents align with regulatory
intent, legal precedents, and ethical guidelines. Furthermore, it provides a
mechanism for addressing exceptional cases that may fall outside the scope of
routine procedures. Because of these concerns various jurisdictions are
developing guidelines on the use of AI in decision making processes (see e.g.,
[7]).

Human oversight also plays a critical role in monitoring LLM agents’
performance, ensuring that they continue to operate effectively and without
bias. Regular audits and reviews by human experts help identify and rectify
any potential issues or biases that may arise in the LLM’s decision-making
process. This synergy between LLM agents and human oversight strikes a balance
between efficiency and regulatory compliance, ultimately leading to more
reliable and fair outcomes.

##  III Challenges and Ethical Considerations

Leveraging LLMs in the government regulatory and policy-making domain holds
significant promise for enhancing efficiency of the processes. However, it
also comes with various challenges and limitations which should be taken into
consideration. In what follows, we will provide a high-level overview of these
challenges.

###  III-A Unconscious Bias

GenAI may amplify existing unconscious biases in the data (e.g., those related
to ethnicity, gender, or other demographic aspects), leading to discriminatory
or non-representative content, potentially harming diversity in ideas and
perspectives [8]. To address these, we need to test for biases in the training
data, as well as model outputs both before deploying a system, and on an
ongoing basis. The process of improving LLMs’ generated output to make it more
consistent with a desired set of values is typically referred to as alignment.
A recent survey of alignment techniques such as Reinforcement Learning from
Human Feedback (RLHF) and Direct Preference Optimization (DPO), where the
reward inferred from human preferences is used to steer LLMs toward generating
better outputs, can be found in [9].

###  III-B Inaccuracy

Output of generative models such as LLMs can be misleading, appearing credible
but being factually wrong (so-called hallucination) [10], or outdated. This in
turn could misinform policies or public communication and lead to loss of
public/stakeholders trust. Therefore, it is important to keep LLMs grounded in
facts by providing them access to authoritative sources of domain knowledge
and engineering prompts that would discourage hallucinations. An example of
this will be demonstrated later in Section IV-B.

###  III-C Automation Bias

Relying too much on AI-generated outputs and decisions can lead to automation
bias where automatically generated outputs are preferred even when faced with
contradictory evidence suggesting other manually-derived outputs might be
better [11]. This can potentially limit creativity and critical thinking of
the domain experts. To address this issue, it is essential for users to
critically assess whether they need generative AI for a task, be aware of the
strengths and weaknesses of this technology, and to use generative AI as a
complement to human judgment instead of a substitute.

###  III-D Legal Risks

Using generative AI such as LLMs in the regulatory and policy-making domains
poses risks to privacy, intellectual property rights, and procedural fairness
due to the data and processes involved. For instance, code that is
automatically generated by an LLM might be a reproduction of existing code
that is not licensed for commercial use. This calls for a thorough risk
analysis of the training data used to train off-the-shelf LLMs and/or
carefully choosing data sources used for training or fine-tuning internal
models.

##  IV Case Studies and Real-World Applications

In this section we give a few specific examples of real-world experimentation
with LLMs to transform some of the radio spectrum regulation processes.

###  IV-A Rules as Code and Knowledge Graphs

In this section we illustrate an example of machine-readable rules extraction
and knowledge graph creation using an LLM. This can be achieved in a multitude
of ways by e.g., providing the related regulatory document(s) to a local or
API-based LLM, or allowing an LLM agent to search the web to find pertinent
document(s) for rule extraction. For illustration, we employ the latter
approach using ChatGPT-4 interface to extract technical rules related to
operation in Personal Communication Systems (PCS) bands in Canada. The prompt
and response are shown in the following where the ChatGPT-4 has found the
related document [12] and then correctly extracted and structured the radiated
power and antenna height limits under different scenarios in machine-readable
JavaScript Object Notation (JSON) format.

PCS Band Technical Parameters Extraction

Prompt: ⬇ What are the radiated power and antenna height limits in the PCS
bands in Canada? Return the answer in JSON format. Response: ⬇ {"PCS Bands
Canada": { "Base Stations Less Equal 1MHz": { "Max eirp": { "HAAT_up_to_300m":
"3280 watts", "Urban_Areas": "1640 watts", "Height_Restrictions": [
{"HAAT_up_to_500m": "1070 watts"}, {"HAAT_up_to_1000m": "490 watts"},
{"HAAT_up_to_1500m": "270 watts"}, {"HAAT_up_to_2000m": "160 watts"} ] } },
"Base_Stations_More_1MHz": { "Max_eirp_per_MHz": { "HAAT_up_to_300m": "3280
watts", "Urban_Areas": "1640 watts", "Height_Restrictions": [
{"HAAT_up_to_500m": "1070 watts per MHz"}, {"HAAT_up_to_1000m": "490 watts per
MHz"}, {"HAAT_up_to_1500m": "270 watts per MHz"}, {"HAAT_up_to_2000m": "160
watts per MHz"} ] } }, "Mobile_Stations": { "Max_eirp": "2 watts" } } }

The output of the LLM can further be used in downstream tasks such as creating
a knowledge graph capturing the relationship between various entities such as
frequencies, radio station type, antenna height above average terrain (HAAT),
and equivalent isotropically-radiated power (EIRP). This process, too, can be
automated via the LLM agent which, this time, leverages its code interpreter
tool to plot the knowledge graph as shown next.  Follow-up Prompt 1: ⬇ Extract
a knowledge graph from these rules and return as an image. Response: Resulting
image is shown in Fig. 1.

Figure 1: Knowledge graph of technical rules for PCS bands in Canada, created
by GPT-4.

It is further possible to create an ontology based on the technical rules
extracted from the documents and datasets. This is illustrated in the next
example.  Follow-up Prompt 2: ⬇ Create an ontology based on the extracted
information. Response: ⬇ Concepts: - BaseStation - UrbanBaseStation -
NonUrbanBaseStation - MobileStation - HAAT (Height Above Average Terrain) -
EIRP (Equivalent Isotropicaly Radiated Power) - Bandwidth Properties: -
hasEIRP: BaseStation -> EIRP - hasHAAT: BaseStation -> HAAT - hasBandwidth:
BaseStation -> Bandwidth - hasMaxEIRP: MobileStation -> EIRP Relationships: -
UrbanBaseStation IS-A BaseStation - NonUrbanBaseStation IS-A BaseStation
Constraints: - EIRP and HAAT have numerical values associated with specific
limits. - Bandwidth is a characteristic that affects the EIRP. - MobileStation
has a maximum EIRP of 2 watts. - UrbanBaseStation and NonUrbanBaseStation have
different EIRP limits based on HAAT.

Distilling regulatory documents and datasets into structured knowledge graphs
provides for better inter-connectivity among disparate data sources and
enables enhanced retrieval of information thereby increasing the utility of
information.

###  IV-B RetrievalQA System

Next, we showcase a retrieval-augmented generation (RAG) based question
answering (QA) system [13, 14] capable of leveraging all of the public and
internal regulatory documents of Canada’s national spectrum regulator. This
type of LLM application, which we will refer to as RetrievalQA hereafter, is
of particular interest in the context of regulation and policy-making as it
grounds the LLM in domain knowledge and authoritative data sources which helps
it produce accurate, factual responses.

Next, we will give an overview of the implemented proof-of-concept RetrievalQA
system and highlight its potential benefits. In Section V we will share some
of the lessons learned during the process.

The central concept in RAG systems is to ground the LLM in facts by providing
it with relevant and accurate context and then prompt it to only draw from the
provided context to respond using a prompt template such as the following:

RAG Prompt Template: ⬇ You are a helpful, respectful and honest assistant. Use
the following context information to answer the user’s question. If you don’t
know the answer, just say that you don’t know, don’t try to make up an answer.
Context: {context} Question: {question}

where {context} and {question} are variables determined at run-time.

In order to find and provide relevant context from a large set of documents, a
retriever is implemented which typically uses semantic search (as opposed to
keyword search). Since semantic meaning can often vary in different parts of
longer documents, text is often split into smaller chunks which are then
tokenized and converted to vector representations (vectors of floating
numbers, referred to as embeddings) using pretrained models, and subsequently
stored in a vector database. Such vector representation allows for effective
semantic search where the user query is embedded using the same embedding
model and is then compared with all text-chunk embeddings within the vector
database. By returning a few of the most relevant document chunks, RAGs have
enabled the use of LLMs within specialized domains without the need to fine-
tune.

Fig. 2 shows a block diagram of our implemented RAG architecture. We employ
the open-source frameworks, LangChain [15] and llama.cpp [16] for
orchestration of the pipeline and retriever, and to interact with the LLM. We
use the Faiss library [17] as our vector database to store the vector
representation of document chunks. We also leverage a locally-hosted version
of the Mistral-7B LLM [18] which comes with the Apache 2.0 license and can be
used without restrictions. Despite being a relatively small 7-billion
parameter model, Mistral has shown good performance in public benchmarks. The
use of a locally-hosted LLM is of particular note as for reasons of privacy
and data sovereignty it may be desirable to avoid using an external LLM. We
use a virtual machine with a single NVIDIA V100 GPU to run the RAG.

Figure 2: Block diagram of the RAG-based question answering system.

For this experiment, the system is provided with approximately $2000$
documents in both PDF and HTML formats which are then parsed, chunked, and
embedded. We then proceed with asking in-domain questions on a range of
spectrum regulatory topics. While we omit sample queries and responses here
due to limited space, our experience so far has shown great promise with the
system being able to answer both high-level as well as very specific questions
on a wide range of regulatory topics such as spectrum auction setup, technical
rules, license conditions, and stakeholder sentiments in consultation
responses. This initial success has generated a great deal of interest
internally and serves as a foundation to build upon.

###  IV-C Wireless License Application

For many wireless applications, the user/operator needs to apply to the
national spectrum regulator for a license before they can operate. The
information required during the application as well as technical parameters
and conditions associated with the license can vary widely depending on
frequency, application, location, or time period which, in turn, renders the
application process complex and error-prone, especially for individuals or
smaller entities. This is an area where the power of LLMs could be leveraged
to facilitate the application process by providing clear instructions to
applicants.

Using OpenAI’s ”Create a GPT” framework, we have built a Wireless License
Application chat-bot for Canada that guides applicants through the application
process by providing clear instructions and helps to ensure higher-quality
data entries. This has been achieved by providing the chat-bot with a document
outlining information required for various license applications and assigning
it the specific role of helping the applicants throughout the application
process. The experimental chat-bot then converses with applicants and converts
the inputs into a machine-readable JSON format, ready for consumption by
internal data bases of the regulator.

##  V Lessons Learned

In what follows, we will share some of the lessons learned during the
implementation of our LLM-based question-answering system.

###  V-A Retrieval Bottleneck

Not surprisingly, the retrieval step can become a performance bottleneck for
RAG systems as even the most advanced LLM can not produce good responses if
the context (i.e., document chunks) provided by the retriever is not relevant,
suitable, or accurate. A variety of techniques have been employed to address
this issue.

The process of document retrieval involves a balance between creating smaller
documents that accurately reflect their content through embedding vectors and
maintaining sufficiently large chunks to preserve context. Expanded context
retrieval, aims to balance these competing needs by using small chunks during
semantic search while returning an expanded context window (including some
neighboring chunks) to the LLM [19]. This provides a broader context for the
model to draw from, increasing the chances of fetching relevant information.

Re-ranking methods involve an additional step where the initially retrieved
documents are sorted again, ensuring that the most relevant pieces of
information are prioritized [20]. This is typically achieved using cross-
encoders, which are capable of scoring pairs of text consisting of the user
query and each of the chunks returned by the retriever.

Furthermore, hybrid search techniques combine traditional keyword-based search
(such as BM25 algorithm [21]) with semantic search capabilities to better
capture the context that might be pertinent to a user’s query. By leveraging
all of these techniques, we were able to noticeably boost the performance of
the retriever, and the overall system. It should also be noted that current
trends in LLM development include larger contexts which are likely to further
improve results.

###  V-B Importance of Metadata

To enable a question-answering system to effectively query both structured and
unstructured data, it is important to ensure that the metadata associated with
tables, databases, documents, and other data sources is accurate and well-
defined. Metadata, such as column names, data types and relationships, and
summary of each table’s or document’s subject, aids in the correct
identification and interpretation of data by the retrieval system and LLMs.
Having proper metadata allows a more seamless interaction between LLMs and
structured databases by enabling the LLMs to create proper queries using
Sequential Query Language (SQL) or other methods as applicable. Conversely,
mistakes or ambiguity in SQL code can lead to incorrect or incomplete answers,
highlighting the critical role of precise query formulation in building a
successful question-answering system using LLMs.

###  V-C Pre-processing of Unstructured Data

Dealing with unstructured data such as PDF documents and web pages can pose a
significant challenge for question-answering systems, as the information is
often embedded within the context of the document’s layout, such as tables,
headings, and paragraphs. In such cases, in addition to providing proper
metadata, steps should be taken to preserve the document structure during the
conversion to machine-readable text format by recognizing and properly
extracting tables, headings, and embedded images or diagrams. This structure
should be further preserved during text splitting/chunking process to ensure
that e.g., tables are not split across different chunks thereby creating
potentially incomplete context. This requires specialized techniques and tools
to handle different unstructured data formats effectively, as overlooking this
step can result in a loss of valuable information which, in turn, can
negatively impact the question-answering system’s accuracy.

###  V-D Human-in-the-Loop

Early prototypes and experiments with LLMs in the spectrum regulation domain
will be potentially prone to inaccuracies and bias which in turn can undermine
trust in the system and jeopardize the successful integration and adoption of
this technology. To address these concerns, having early feedback from human
experts can provide a critical layer of oversight. Specifically, domain
experts can verify the accuracy of LLM-generated responses, ensure compliance
with regulations, and provide valuable feedback to fine-tune the system before
fully integrating it into the existing internal or public-facing operations.
This human-in-the-loop approach is essential not only for improving the
performance and fairness of LLMs but also for upholding the integrity and
trustworthiness of regulatory processes in an increasingly AI-driven era.
Furthermore, government guidelines may require human participation [7].

##  VI Conclusion and Future Prospects

The use of AI in general, and LLMs in particular, in the spectrum regulatory
and policy-making domain holds significant promise for enhancing efficiency of
various processes. Automation of routine, yet time-consuming and labour-
intensive tasks via LLMs can free up human resources for more strategic and
creative work and lead to streamlined administrative processes and cost
savings. In this paper we discussed how LLMs can assist in the analysis of
regulations and policies, extraction of rules as code, summarizing and
interpreting stakeholder positions, finding specific spectrum licence
conditions, and other key regulatory workflows. Future work should look at
approaches for ensuring transparency and fairness in LLMs in this context as
well as potential gains from models fine-tuned on spectrum regulatory domain
data.

It should be noted that the human element remains essential to provide
oversight, address exceptional cases, and ensure that the LLM agents operate
in accordance with regulatory intent and ethical standards. As technology
continues to advance, the integration of LLMs into regulatory workflows offers
a promising avenue for enhancing the efficiency and effectiveness of radio
spectrum regulation while upholding the necessary checks and balances.

## References

  * [1] A. Vaswani et al., ”Attention Is All You Need”, _in Proc. 31st Conference on Neural Information Processing Systems (NIPS 2017)_ , pp. 6000–6010, Long Beach, USA, December 2017.
  * [2] J. Mohun and A. Roberts, ”Cracking The Code: Rulemaking For Humans And Machines”, _OECD Working Papers on Public Governance_ , no. 42, available online: https://doi.org/10.1787/3afe6ba5-en, October 2020.
  * [3] OpenFisca, Code available online: https://github.com/openfisca.
  * [4] Z. Zhu et al., ”Large Language Models can Learn Rules”, available online: https://arxiv.org/pdf/2310.07064.pdf, October 2023.
  * [5] W. Huang, P. Abbeel, D. Pathak, I. Mordatch, ”Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents”. _in Proc. 39th International Conference on Machine Learning_ , pp. 9118–9147, 2022.
  * [6] S. Yao et al., ”ReAct: Synergizing Reasoning and Acting in Language Models”, available online: https://arxiv.org/pdf/2210.03629.pdf, 2022.
  * [7] Government of Canada, ”Directive on Automated Decision Making”, available online: https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592.
  * [8] I. O. Gallegos et al., ”Bias and Fairness in Large Language Models: A Survey”, available online: https://arxiv.org/pdf/2309.00770.pdf, September 2023.
  * [9] T. Shen and C. Liu and X. Wu et al., ”Large Language Model Alignment: A Survey”, avilable online: https://arxiv.org/pdf/2309.15025.pdf.
  * [10] S. Hanneke, A. T. Kalai, G. Kamath, C. Tzamos, ”Actively Avoiding Nonsense in Generative Models”, _in Proc. 31st Conference On Learning Theory_ , pp. 209-227, 2018.
  * [11] H. Ruschemeier, ”The Problems of the Automation Bias in the Public Sector – A Legal Perspective”, _in Proc. Weizenbaum Conference_ , available online: https://ssrn.com/abstract=4521474, 2023.
  * [12] Innovation, Science and Economic Development Canada, ”SRSP-510 — Technical Requirements for Personal Communications Services (PCS) in the Bands 1850-1915 MHz and 1930-1995 MHz”.
  * [13] P. Lewis et al., ”Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, _in Proc. 34th Conference on Neural Information Processing Systems (NeurIPS)_ , Vancouver, Canada, 2020.
  * [14] W. Shi et al., ”REPLUG: Retrieval-Augmented Black-Box Language Models”, available online: https://arxiv.org/pdf/2301.12652.pdf, 2023.
  * [15] LangChain, https://github.com/langchain-ai/langchain.
  * [16] llama.cpp, https://github.com/ggerganov/llama.cpp.
  * [17] Facebook AI, ”Faiss: A Library for Efficient Similarity Search”, https://github.com/facebookresearch/faiss.
  * [18] A. Q. Jiang et al., ”Mistral 7B”, available online: https://arxiv.org/abs/2310.06825, October 2023.
  * [19] LangChain, ”Parent Document Retriever”, https://python.langchain.com/docs/modules/data_connection/retrievers/parent_document_retriever.
  * [20] Sentence Transformers, ”Retrieve and Rerank”, https://www.sbert.net/examples/applications/retrieve_rerank/.
  * [21] S. Robertson and H. Zaragoza, ”The Probabilistic Relevance Framework: BM25 and Beyond”, _Foundations and Trends in Information Retrieval_ , vol 3, no. 4, pp. 333–389.