# A renewable power system for an off-grid sustainable telescope fueled by
solar power, batteries and green hydrogen

Isabelle Viole , Guillermo Valenzuela-Venegas11footnotemark: 1, Marianne
Zeyringer and Sabrina Sartori
Department of Technology Systems (ITS)
University of Oslo
Kjeller, Norway
isabelle.viole@its.uio.no(IV<EMAIL_ADDRESS>(GVV) These
authors contributed equally to this work.

###### Abstract

A large portion of astronomy’s carbon footprint stems from fossil fuels
supplying the power demand of astronomical observatories. Here, we explore
various isolated low-carbon power system setups for the newly planned Atacama
Large Aperture Submillimeter Telescope, and compare them to a business-as-
usual diesel power generated system. Technologies included in the designed
systems are photovoltaics, concentrated solar power, diesel generators,
batteries, and hydrogen storage. We adapt the electricity system optimization
model highRES to this case study and feed it with the telescope’s projected
energy demand, cost assumptions for the year 2030 and site-specific capacity
factors. Our results show that the lowest-cost system with LCOEs of $116/MWh
majorly uses photovoltaics paired with batteries and fuel cells running on
imported and on-site produced green hydrogen. Some diesel generators run for
backup. This solution would reduce the telescope’s power-side carbon footprint
by 95% compared to the business-as-usual case.

_Keywords_ Power system optimization $\cdot$ Hybrid energy storage $\cdot$
Off-grid $\cdot$ Green hydrogen $\cdot$ Carbon footprint

##  1 Introduction

To contribute to the global effort limiting the global rise in temperature
below 1.5∘, astronomers need to reduce their carbon footprint. Today’s carbon
footprint per astronomer is estimated at ~37t CO2 equivalents (CO2e) annually
[1, 2]. The sector as a whole emits ~1,200kt CO2e per annum, that is 0.0024%
of the global greenhouse gas (GHG) emissions (50.2Gt in 2020). This exceeds
the population share of this group by more than a factor of six, since just
30,000 out of the current population of 8bn are astronomers [2]. Next to CO2e
emissions related to flights and powering supercomputers, 13% stems from
powering the operation of observatories [1]. These have high power needs for
the cryogenic cooling of instruments and the motors tracking their dishes. As
most of this electricity stems from fossil fuel sources today, ground-based
telescopes have a median carbon intensity of 24t CO2e per paper published
using their generated data [2]. For comparison, the carbon footprint per
capita in the European Union was 6.8t CO2e in the year 2019 [3].

Growing awareness of climate change mitigation and volatile prices of fossil
fuels led some observatories to subsequently add Renewable Energy Sources
(RES) to their power generation. In Hawaii, the Gemini Observatory supplies
10% of its demand with solar photovoltaic (PV) arrays since 2015. In the
Chilean Atacama Desert, the La Silla observatory is powered by more than 50%
solar energy since 2016 and the Paranal observatory commissioned PV arrays in
2022 [4, 5, 6].

Due to the decades-long lifetime of astronomical research infrastructures,
decisions taken on power supply in new projects today lock in CO2e emissions
for decades. To strive for lower carbon footprints, AtLAST - The Atacama Large
Aperture Submillimeter Telescope - is the first observatory already including
plans for a power system powered by RES in its design stage [7]. AtLAST is a
design study project for a 50m single dish submillimeter/millimeter telescope
on the Chajnator plateau in the Atacama, at an elevation of ~5,000 meters [7,
8]. It is planned to start its operation in the early 2030s. The very high and
dry site is necessary to minimize water vapor in the atmosphere, which absorbs
submillimeter photons. The eleven neighboring telescopes, like the Atacama
Pathfinder Experiment (APEX) and the Atacama Large Millimeter/ submillimeter
Array (ALMA), today rely on fossil fuels to meet their power demands.

Several recent studies carried out simulations and techno-economic assessments
of islanded stationary systems for household demands lately, considering
present-day component costs. Endo et al. [9] simulated the operation of a
stationary system to power a residential building, consisting of RES, an
electrolyzer, a hydrogen (H2) storage system in metal hydrides (MH), a proton-
exchange membrane fuel cell (PEMFC) and batteries. They showed that PV and
PEMFC generation can meet most of the demand during fine and cloudy weather.
Ghenai et al. techno-economically optimized an off-grid solar PV and H2 energy
system to meet the 500kWh daily demand of a desert community, resulting in
levelized costs of electricity (LCOE) of $145/MWh [10]. Gebrehiwot et al.
similarly analyzed a hybrid power system with PV, wind, and diesel generation
plus batteries to cover a remote village’s demand, LCOEs resulting in $207/MWh
[11]. Dawood et al. compared off-grid systems with PV, batteries and/or H2
storage for a remote settlement demanding 2MWh daily, where a hybrid energy
storage system with both batteries and H2 induced lowest costs of electricity
with $342/MWh. This more than halved the PV curtailment compared to coupling
PV only with batteries [12].

We identify a number of research gaps in literature: First, to our knowledge,
there is no similar study analyzing the design of a renewable energy system
for a remote research facility in general and telescope in particular. Due to
very different needs as well as environmental conditions study results from
residential settings can’t be applied to telescopes. Further, studies on
islanded, renewable-based energy systems have not been applied to high
altitudes yet, only use a limited number of technologies and do not consider
technology learning.

In this work we close these gaps by: 1\. performing the first optimization of
a power system for a remote telescope with a unique electricity demand,
characterized by seasonal shifts and minimal day-night fluctuations. It serves
as a pioneering example for the transition to RES in off-grid research
facilities, crucial for reducing CO2 emissions to meet climate targets. 2\.
considering elevation-specific derating factors serving as a first study in
the design of high-altitude power systems 3\. providing a more comprehensive
comparison of system setups based on renewable energy for remote and off-grid
power systems: We evaluate the trade-offs between 100% renewable systems, and
systems with fossil fuel backup, next to trade-offs between batteries and
hybrid energy storage systems, offering insights into the most cost-effective
and sustainable solutions. We compare systems using imported and on-site
produced green H2 to those only employing batteries. Concentrated Solar Power
(CSP), a promising technology in high irradiation places, is compared to the
more widely applied PV technology. 4\. applying technology learning rates in a
sensitivity analysis: In line with the projected start of AtLAST’s operations
in the early 2030s, we size a power system for 2030 with a wide range of cost
decline assumptions in RES and varying fuel costs. In contrast to modeling the
system with today’s costs, the use of projected future costs paired with a
cost-sensitivity analysis helps to better showcase relevant energy systems for
AtLAST in the next decade in which it will need to be supplied. It thereby
also represents a valuable case study for similar, remote infrastructure
projects that are currently being planned.

Specifically, this work presents solutions to power AtLAST’s consumption of
annually ~7.7GWh with an isolated power system, including business-as-usual
powering with fossil fuel generation, and combinations of RES, energy storage
and fossil fuel generation backup. We answer the following research questions:

  1. 1.

Which system results in the lowest levelized costs of electricity (LCOE) to
power the planned telescope?

  2. 2.

How much more does it cost to use a power system without direct
CO2e-emissions?

  3. 3.

Which system setups are robust against uncertainty in future costs?

##  2 Materials and Methods

We apply a cost-optimization model to find feasible power systems for the
telescope, divided into eight scenarios:

  1. 1.

Using diesel generators at 5,000m (Business-as-usual, BAU);

  2. 2.

Combining PV and diesel generation (PV & Diesel, PVD)

     * •

at 5,000m (PVD↑);

     * •

at 2,500m (PVD↓);

  3. 3.

Combining PV generation with a hybrid energy storage system

    1. (a)

with backup diesel generation (PV, Diesel & Energy Storage, PVDES)

       * •

at 5,000m (PVDED↑);

       * •

at 2,500m (PVDES↓);

    2. (b)

as a renewables-only system without diesel generation (Renewable Energy
Sources, RES)

       * •

at 5,000m (RES↑);

       * •

at 2,500m (RES↓);

  4. 4.

Using CSP at an altitude of 2,500m (CSP).

2030 is set as the potential year to start operation. Location-wise, we
consider systems built near the telescope on the Chajnator plateau at ~5,000m
and system at a Valley Site, 2,500m above sea level, see Figure 1. For the
telescope’s location, currently two options less than 5km apart from each
other are investigated. These are undistinguishable irradiation-wise, both
present temperature variations from -20 to 10°C and low pressures of ~550Pa,
for which we apply derating factors, and hence are considered alike in this
work. The Valley Site, 43km away from the telescope, implies less derating of
components and easier access for operations, while adding costs for
subterranean cabling. Only off-grid systems are considered.

Figure 1: Map of the studied area, highlighting a potential sites of AtLAST,
the power generation location in the valley and the necessary power line
pathway between the two.

To optimize the different scenarios, we apply highRES-AtLAST, an adaptation of
the highRES model proposed by Price and Zeyringer for the energy system of
Great Britain and Europe [13, 14]. Constructed to design a cost-minimal power
system using a linear programming model, it minimizes overall costs while
meeting hourly demands and technical constraints. The main outputs are system
costs, built capacities and hourly dispatch. We add a H2 storage system with
electrolyzers, storage and fuel cells and CSP as technologies to the model.
The H2 and CSP system (thermal energy) were implemented in terms of power (MW
and MWh) to fit with the highRES limitation. The limitations of highRES-AtLAST
are the same as the original model. Figure 2 shows an overview of all
technologies in the system. 2030 serves as the demand and cost assumption
year, while costs are set to inflation-adjusted real2022-US$ [15]. Inputs for
the model include solar generation capacity factors, Chilean component and
fuel costs in 2030 and a demand time series for AtLAST. Different learning
rates for component costs and variations in fuel costs are integrated via
sensitivity analyses.

Figure 2: Potential components of power system for the telescope, inverter not
included in optimization.

###  2.1 Estimating the telescope’s demand

The demand of AtLAST comprises dish motors, a cryogenic cooling system for the
instruments, electronics, instrument support, and a heating, ventilation, air
conditioning system, see Table 1. Peak demands are supplied by supercapacitors
outside the system presented. For the optimization, the historical demand of a
smaller radio telescope is upscaled to match AtLAST’s estimated demand,
resulting in a one-year time series for 2030 at hourly granularity. Figure 3a)
shows the estimated average monthly power demand, see Figure 3b) for exemplary
daily demand curves.

Table 1: Demand estimation of AtLAST. Demand constituent | Average demand (kW) | Peak demand (kW)
---|---|---
Cryogenic cooling | 450 | 600
Telescope motors | 350 | 1300
Heating, ventilation and air conditioning system | 40 | 50
Electronics; instrument support; other | 40 | 50
Figure 3: Demand patterns of AtLAST in 2030, a) average monthly demand; b)
exemplary daily demand curves.

###  2.2 Choice of system components

The global solar irradiation maximum lies within the Atacama Desert, an annual
irradiation of ~2,640kWh/m2 can be assumed at the prospective sites [16, 17].
As inputs for the optimization, we convert hourly irradiation values from
ERA5, previously used to represent the weather conditions on a regional level
in modelling the Chilean electricity generation [18], into capacity factors
[17, 19], see Figure 4. While PV systems can be located at either 5,000m or
2,500m, CSP is excluded at the high site, to not interfere with astronomical
observations. Wind power is not considered, as the areal average wind speed of
3.1m/s is too low for power production [20].

Figure 4: Solar generation capacity factors: (a) PV at AtLAST Site, (b) at
Valley Site, (c) CSP ST at Valley Site, (d) CSP PT at Valley Site.

To represent CSP in the model we consider three interlinked subsystems: the
solar field (capturing solar irradiation), the thermal energy storage (storing
surplus thermal energy), and the power block (generating power) [21]. We
consider the two most established CSP technologies, the Parabolic Trough (PT)
and the Solar Tower (ST) [22]. Advantages lie in their flexibility, as thermal
energy storage allows for power generation on demand. However, CSP is often
not competitive with the steeply declining PV costs, so some systems have been
retrofitted to PV [23].

Today’s telescopes in the area are for the majority supplied by diesel
generators, which can ramp up and down flexibly and are dependent on regular
refueling trucks. We employ similar diesel generators in our model.

####  2.2.1 Energy storage systems

We consider both batteries and H2 as energy storage technologies. In
batteries, mature options include lead acid and lithium-ion (Li-ion)
batteries. According to manufacturers, both technologies can withstand
temperatures down to -20°C in discharge/storage, and down to 0°C when
charging. Lead acid batteries with their historically lower investment costs
currently hold the leading role in stationary systems. In recent years,
however, some found lower LCOEs with Li-ion batteries, based on their higher
cycle life [24]. As additionally, Li-ion batteries perform better in life
cycle assessments when compared to lead acid, we include only this technology
in our setup [25].

When utilizing H2 as an energy carrier in remote systems, two ways of
operation are thinkable. For one, H2 production off-site and shipment to the
location is doable, requiring on-site storage and fuel cells to generate
power. We only consider so-called green H2, stemming from RES-powered
electrolysis. Without imported H2, the whole H2 value chain is needed on-site,
that is electrolyzers, H2 storage and fuel cells. H2 storage in stationary
systems today is mostly conducted with compressed gas (CG) stored in steel
cylinders at 50-700 bar [26]. MH applications for storing H2 are investigated
as a safer alternative to CG, e.g. by Endo et al. [9], but are not
economically available yet.

####  2.2.2 Derating factors

We do not know of any solar power system with a substantial energy storage
system as proposed in this work in existence at 5,000m. In this techno-
economical assessment, we consider technologies that could endure the harsh
conditions of the location, and include derating factors to consider the low-
pressure conditions at either site, see Table 2. These factors impact the
components’ power output. For instance, a diesel generator with a capacity of
2MW at the ATLAST Site has a maximum effective capacity of 1MW, that is its
capacity multiplied with a derating factor of 0.5. Diesel generators are
impacted by the less dense air, as less oxygen molecules are available for the
burning process. Batteries are derated at altitudes, as the extremely low
temperatures and low pressures represent an issue for their liquid
electrolyte. Both electrolyzers and fuel cells are impacted by the lower
amount of oxygen molecules available. The factors of diesel generators and
batteries stem from empirical data and internal communications with experts
from other telescopes in the areas. In experiments of Pratt et al., a PEMFC at
an altitude of 2,774 meters had 6-10% less stack voltage than its counterpart
at an altitude of zero [27]. We hence assume similar derating factors for H2
and batteries.

Table 2: Derating factors at 2,500 and 5,000 altitude meters. System component | Derating factor at 2,500m | Derating factor at 5,000m
---|---|---
Diesel generator | 0.75 | 0.5
Electrolyzer/Fuel Cell/Li-ion battery | 0.95 | 0.8

###  2.3 Component and fuel costs, sensitivity analysis

PV, batteries, and H2 systems had a steep learning curve over the last
decades, a trend forecasted to continue until beyond 2030, the planned
building year of this system. In this work, we use Chilean forecasted costs,
wherever possible, and integrate different learning rates in sensitivity
analyses to account for variations in future component costs. The base case
capital expenditures (CAPEX) and lifetime assumptions are shown in Table 3,
with costs adjusted to real2022-US$ values. See Table A.1 in Appendix for
operational expenditures (OPEX). This optimization includes generation,
storage units and long-distance cabling as the major cost components. Other
electrical system components like inverters are not integrated, though they
need to be considered in any final system design. Sensitivity analyses with
costs as indicated in Table 4 and Table 5 are performed to indicate the
robustness of system setups against different learning curves and other fuel
costs than the ones assumed in the base scenario.

Table 3: Cost estimations in 2030, real2022-US$ values. Cost component | CAPEX | Unit | Lifetime
---|---|---|---
Monofacial monocrystalline PV | 523 [28, 29] | $/kWp | 25 years
CSP solar field – PT/ST | 209/148 [30] | $/m2
CSP thermal energy storage – PT/ST | 37/19 [23, 30, 31] | $/kWhth
CSP power block – PT/ST | 869/1,156 [30] | $/kWe
Diesel generator | 495 [32] | $/kW
Li-Ion batteries | 262 [33, 34] | $/kWh | 13 years [33]
Electrolyzer - Alkaline / PEM | 434/483 [34] | $/kWe | 18 years [33]
CG H2 storage | 598 [35] | $/kg H2 stored
PEMFC | 1,581 [36] | $/kWe
Subterranean power lines, 24kV | 23,599 [37] | $/km | 40 years
Table 4: Sensitivity case cost estimations in 2030, real2022-US$ values. Cost component | Low case CAPEX | High case CAPEX | Unit
---|---|---|---
Monofacial monocrystalline PV | 345 [28, 29] | 928 [38] | $/kWp
Li-ion batteries | 164 [33] | 304 [34] | $/kWh
Electrolyzer - Alkaline / PEM | 247/327 [39] | 774 [38] | $/kWe
CG H2 storage | 286 [35] | 780 [35] | $/kg H2 stored
PEMFC | 1,002 [40] | 1,876 [36] | $/kWe
Table 5: Fuel costs in 2030, base case and high-cost sensitivity, real2022-US$ values. Fuel | Base case costs | High case costs | Unit
---|---|---|---
Diesel | 600.76 [41] | 1,220.52 [42] | $/m3
Green H2 | 1.13 [43] | 1.40 [36] | $/kg
Transporting H2 | 0.54 [44] |  | $/kg/100km

####  2.3.1 Power generation costs

In 2020, the median costs of Chilean PV systems >1MWp was at
$\$_{2020}$769/kWp (real2020-$, inflation-adjusted US dollar value to the year
2020) [28]. To forecast future costs we apply Vartiainen et al.’s base PV cost
decrease rate of 39.3% from 2020 to 2030 on the Chile-specific costs,
resulting in $515.90/kWp in 2030 [28, 29]. This is comparable with the
International Renewable Energy Agency (IRENA)’s (2019) assumptions of
$\$_{2019}$340-834/kWp for utility-scale PV plants in 2030 [38]. For
sensitivity cases, we include IRENA’s slow learning rate as our high, and the
fast learning rate of Vartiainen et al. with a cost decrease of 59.4% as our
low case sensitivity. See Table 4 for sensitivity case assumptions. Figure 5
a) shows the cost trajectories for PV and CSP.

Figure 5: Cost trajectory from 2020 to 2030 a) for solar power generation, PV
margins for cost sensitivities; b) for energy storage components, margins for
cost sensitivities.

With CSP, several studies report learning rates for this technology, though
few present results based on future scenarios. The Joint Research Centre
forecasts a learning rate of 33% from 2015 to 2030 [45]. The Chilean Energy
Ministry projects CSP cost decline of 23% in its carbon neutrality scenario
[46]. Slower and faster learning rates range from 5-49%. We adopt this Chilean
carbon neutrality learning rate in our optimization, modularizing it on
individual component costs from the National Renewable Energy Laboratory
(NREL) [30]. CSP system costs in this work exceed those of PV and diesel-
systems by far, so no sensitivity analysis is undertaken.

####  2.3.2 Energy storage costs

By 2030, Li-ion costs are assumed to decline from 2020’s $\$_{2020}$350 to
$\$_{2020}$270/kWh or $\$_{2020}$148-250/kWh by the the US-American Department
of Energy (DOE) and NREL, see energy storage cost declines in Figure 5 b) [34,
33]. For our base case, we average between the medium NREL projections DOE’s
$304/kWh, setting the value at $262/kWh. The lower end of NREL’s projection
serves as the low and the DOE’s normal assumption as the high sensitivity.

H2 system cost decline projections are changing rapidly. In 2019, the
International Energy Agency (IEA) assumed a technology-neutral decline in
electrolysis costs from $\$_{2019}$900/kWe down to $\$_{2019}$700/kWe in 2030
[47]. The DOE (2020) assumed PEM electrolyzers costs to go down to
$393-481/kWe by 2030 [34]. In 2021, the Hydrogen Council and McKinsey &
Company forecasted a decline from $\$_{2021}$550-1,050/kWe towards
$\$_{2021}$230-380/kWe in 2030 [39]. Industrial electrolyzer suppliers
contacted by the authors expressed that CAPEX they see with vendors beat the
recent literature predictions. We use the DOE’s mean value as our base case at
$486/kWe, the Hydrogen Council’s mean value of $327/kWe as the low
sensitivity, and the IEA’s $774/kWe as the high sensitivity case.

The more mature alkaline electrolyzer technology has lower investment costs
than their PEM counterparts today, though Saba et al. foresee the costs of PEM
systems approach those of alkaline electrolyzers soon [48]. We set base
alkaline electrolyzer costs at the lower end of the DOE’s PEM electrolyzer
cost forecasts at $434/kWe [34], and follow the lowest value of the Hydrogen
Council for the low, the IEA’s normal cost assumption for the high-cost
sensitivity.

In CG H2 storage systems, Sens et al. report costs of $286-$780/kg H2 in 2030
[35]. Their mean value serves as our base case for CG, the lower and upper
boundaries as the sensitivity costs. In stationary PEMFC costs, the European
Union’s Clean Hydrogen Partnership (2019) announced target cost declines from
$\$_{2019}$1,900/kWe in 2020 to $\$_{2019}$900/kWe in 2030 [40]. Cigolotti et
al. (2021) expect cost drops from $\$_{2021}$2,000-3,500/kWe to
$\$_{2021}$1,200-1,750/kWe by 2030 due to economies of scale [36]. We set
their mean assumption as our base case. In the sensitivity analysis, the Clean
Hydrogen Partnership’s target serves as the low, with Cigolotti et al.’s upper
value as the high case.

Uncertainties in the energy storage system comprise whether a big battery park
is functional at the altitudes discussed and whether a green H2 economy will
develop until 2030. To find meaningful power system designs for AtLAST, we
compare systems with hybrid energy storage against designs without a battery
park or without H2.

####  2.3.3 Line and fuel costs

For Valley Site power systems, transmission lines need to be built to reach
the ATLAST Site. We estimate 43km of newly built 24kV subterranean lines
between the generation site and the telescope. Overhead lines are not
applicable due to the sulfur-rich environment and harsh weather conditions. We
apply costs from a Chilean contractor for subterranean lines plus a 30%
surcharge for the remote and harsh conditions [37].

For the option of using off-site produced H2, we follow the Chilean
government’s (2021) assumption of $\$_{2021}$1.05 per kg of green H2 produced
in 2030, based on analytics by McKinsey & Company [49]. IRENA (2022) expects
costs of green H2 in Chile of $1.4/kg H2 in 2030, which shall serve as our
high sensitivity [42]. Following the work of Tashie-Lewis and Nnabuife (2021),
we consider $\$_{2021}$0.50/kg/100km for transporting H2 in trucks [50]. We
assume to import H2 from the Pauna Solar park in María Elena weekly [51, 44].
The trucking distance to the ATLAST Site is 250km, to the Valley Site 200km.
All fuel costs are listed in Table 5.

With diesel, we are subject to volatile fossil fuel prices. 2022’s peak
Chilean diesel price was at $1220.52/m3, nearly double the previous 4-year
average of $\$$566.28/m3 [52]. In private communications with the authors,
Rystad Energy forecasted the Brent Crude Oil price to decline by 103% from
2022’s high of $122.71/b to $60.40/b in 2030 [52]. Applying this ratio to the
Chilean diesel price, we result in $601/m3 in 2030 in our base case. We
include 2022’s peak as the high sensitivity case. Shipping of diesel is
included in the diesel generator OPEX, see Table A.1.

##  3 Results

The model highRES-AtLAST optimizes total system costs (annualized investment
and dispatch costs). The results include the built capacity, dispatch, direct
CO2e emissions from operation and the total electricity dispatch costs,
amongst others. To calculate LCOEs of the system, the latter is divided by the
annual demand of 7.7GWh:

$\text{LCOE}=\frac{\text{Annualized total electricity system
cost}}{\text{Annual demand}}.$

In the following section, we discuss the base case scenario optimizations. We
look at changing results when one energy storage option is excluded and
further dive into costs from sensitivity analyses.

###  3.1 Base case optimized power systems

Figure 6 shows the LCOEs, annual generation and built capacity of power
generating units of the optimized scenarios, assuming costs as in our base
case. See Table A.2 in Appendix for an extensive overview of the results.
Scenarios with PV generation achieve LCOEs between $116.0 and $144.6/MWh.
Employing PV together with energy storage solutions (PVDES and RES) results in
lower generation costs than supplying the demand not covered by PV mostly with
diesel generators (PVD). The lowest costs are achieved in PVDES, where the
power system consists of PV, batteries, H2 and diesel for backup. The PVDES
and RES systems produce 51-56% of their H2 supply on-site, the rest is
imported. When diving into the hourly generation data, we see that batteries
and PEMFC generation are used throughout the nights of the simulated year, see
Figure 8. Power generation from diesel is mostly called during periods of less
sunny days. The Chilean summer months January to March have enough solar
generation and storage capacity to meet the seasonally lower telescope demand,
while the autumn and winter months use generation from diesel for cloudy
periods. When cutting out diesel generators in the optimization, LCOE values
rise by 6-9 percentage points (pp.), including more generation from green H2.
The CSP scenario results in LCOEs of $379.5/MWh.

Figure 6: Results base case: (a) Annual generation in GWh and LCOEs in $/MWh,
(b) built capacity in MW.

To evaluate the direct carbon footprint associated with the energy system in
each scenario, we considered that only the diesel generator emits GHG during
its operation. Assuming an emission factor of 0.86t CO2e/MWh [gaete-
morales_assessing_2018], the direct GHG emissions, measured in CO2e, are
highest in BAU, see Table 6. Pairing diesel with PV reduces the annual
emissions by about 40%, while PVDES scenarios cut 93-95% of the diesel-only
emissions. Scenarios without diesel generators forego any direct emissions.

Table 6: Direct CO2e emissions of the base case optimizations. Scenario | BAU | PVD↑ | PVD↓ | PVDES↑ | PVDES↓ | RES↑ | RES↓ | CSP
---|---|---|---|---|---|---|---|---
Annual emissions |  |  |  |  |  |  |  |
(t CO2e/year) | 6,624 | 3,892 | 3,954 | 465 | 323 | 0 | 0 | 0
CO2e reduction |  |  |  |  |  |  |  |
compared to BAU (%) | – | 41.20% | 40.30% | 93.00% | 95.10% | 100% | 100% | 100%

###  3.2 Systems without battery park or without green hydrogen

Figure 7 compares results from the base case PVDES and RES scenarios without a
battery park and without H2. In cases without a battery park, resulting power
systems mainly consist of PV and green H2. LCOEs range $121-124/MWh with
diesel as backup, and $129-135/MWh without. A small capacity of batteries is
built to balance frequency and voltage, assumed at 0.01MWh. The resulting
systems in median produce 62% of the needed H2 on-site. In the H2-based RES
scenarios, an LCOE increase of 6pp. must be taken to reduce the final 13% of
CO2e emissions from PVDES. The size of PV parks increases by 27-42% compared
to the base case with batteries, as more PV generation fuels electrolyzers.
When cutting out H2 as an option, results from the PVDES and RES scenarios
change as shown in Figure 7c. We see setups based on larger PV and battery
capacities. The PV capacity in the RES scenarios increases by 28-32%; with
44-49% of its generation curtailed. LCOEs lie ~$117/MWh when diesel serves as
backup, and $139-145/MWh without diesel in the system. In the RES scenarios
without H2, LCOEs increase by 10pp. compared to their PVDES counterpart. This
saves the last ~8.6% of CO2e emissions which PVDES without H2 emits.

Figure 7: Annual generation in GWh and LCOEs in $/MWh: (a) Base case with
hybrid energy storage, (b) base case without battery park, (c) base case
without green H2. Figure 8: Power generated and battery charging/discharging
over optimized year, scenario PVDES↑: (a) PV generation, (b) battery charging,
(c) battery discharging, (d) PEMFC generation, (e) diesel generation.

###  3.3 Sensitivity analyses

Optimizing the power system with the sensitivity cost assumptions as in
Section 2.3 results in varying system setups and costs. The resulting LCOEs
are shown in Figure 9, with the values resulting from the base case costs as
black diamonds. In the H2 systems high sensitivity, high costs of
electrolyzers, fuel cells and imported green H2 are applied coincidently. In
the low case, low costs of electrolyers and PEMFC are used with base case
green H2 costs.

Higher fossil fuel costs lead to hefty LCOE jumps of 83% in the BAU scenario.
Similar jumps of 70% are present in the PVD scenarios with high diesel costs,
while PVDES only sees a cost increase of 3%. The scenarios with energy storage
options, PVDES and RES, respond most to higher PV costs on the upper end. On
the lower end, lower battery and lower H2 system costs make the biggest
difference on them.

Figure 9: LCOE comparison, base case costs (black diamonds) and sensitivity
cost assumptions.

##  4 Discussion

All scenarios calculated result in systems that can cover the forecasted
annual demand of the telescope AtLAST, as all optimizations converge to an
optimal solution. Obtained LCOEs in the base case lie $116-201/MWh, that is
below up to a similar range to those of stationary systems for residential
demanders discussed in Section 1. Lower generation costs in the power system
setups presented are plausible given the cost decline assumptions applied, and
pointing to the global solar maximum in the Atacama. Regarding our research
questions, we find:

  1. 1.

The base case cost assumptions lead to low LCOEs in PVDES and RES scenarios,
with most of the generation carried by PV and batteries, next to some
H2-driven fuel cells. Lowest costs with $116/MWh are achieved in PVDES, with
generators running on diesel used for cloudy days.

  2. 2.

An all renewable power system as in RES↑ has LCOEs increase by 6% compared to
PVDES↑. This small spike avoids the final 7% of the BAU scenario’s emissions,
equaling 465t of CO2e, or 183 European researchers flying roundtrip to AtLAST
(see CO2e emissions calculation with the Environmental Footprint 3.0 Method in
Supplementary Material).

  3. 3.

In the sensitivity analyses with upper and lower boundaries of expected costs
in 2030, see Section 2.3, a jump above the base case cost value window of
$116-201/MWh presents itself in the diesel-only scenario. When applying the
peak Chilean diesel price, system costs increase by 100% compared to the base
case. It is deduced that a power system solely dependent on a fossil source of
power generation leads to fickle power generation costs. We propose to employ
a system with PV and applicable energy storage to power AtLAST sustainably,
both in terms of GHG emissions, but also more robust costs.

GHG emissions related to the operation of the power system in scenarios
swerving from the BAU approach reduce CO2e-emissions by 40 to 100%. PEM
electrolyzers are preferred over alkaline ones, CSP ST is preferred over CSP
PT. Systems that employ H2 both use imported and on-site produced H2. The
import is often used to cover H2 demand peaks when high power demand meets
cloudy weather.

Both systems without H2 or without batteries were simulated to consider cases
where these technologies cannot be considered for the telescope. This could
e.g. occur in case the electrolyte of batteries does not operate well at the
altitude, or in case green H2 is not widely available as a fuel in 2030. Both
instances result in more usage of diesel generation in PVDES, as seen in
Figure 7b and c. Not having batteries is slightly more expensive than not
having H2. In RES, more PV capacity is needed to either supply a bigger
battery park or electrolyzers. Decarbonifying the battery-rich systems without
green H2 results in high PV curtailments of up to 40%. Scenarios relying on PV
and storage solutions generally resulted in lower costs than diesel-relying
scenarios BAU and PVD, regardless whether batteries, green H2 or a combination
of the two are used. Low-CO2e emitting systems in PVDES need to install less
PV and storage capacity than 100% renewable scenarios, resulting in lower
costs and less curtailment.

Comparing PV and CSP systems, CSP presented costs of $380/MWh, vastly higher
than any PV scenario. With CSP costs forecasted as in this paper, this
technology is not an economically viable option for AtLAST. To study the
feasibility of this solar technology in larger systems, one could include
additional demands, like neighboring telescopes or a nearby city, in future
case studies.

###  4.1 Weather data, climate change reliability

The solar capacity factors in this work were based on solely two ERA5 squares
of 30km2. With altitude changes from 2,500m to 5,000m, considerable variations
in the weather conditions are expected. These cannot be represented within a
single square grid, as the irradiation within is considered uniform. To better
estimate the solar generation at the AtLAST and Valley Site, future work
should consider the use of a higher resolution database.

For solar generation, we assumed the weather conditions for 2030 as similar to
2020. While estimations of solar variation expect the Chajnator area to
undergo no high changes in irradiation until 2065 [23], these small changes
can be propagated during the system’s lifetime and affect its reliability. To
design a reliable power system and reduce the uncertainty associated to
weather data, a future climate scenario database could be considered in
further investigations.

###  4.2 High altitude considerations

This work considered both power systems next to the new telescope at an
altiutude of ~5,000m and systems down in the valley at ~2,500m, connected to
the telescope via medium-voltage lines. The altitude made little difference to
the resulting system costs, so that for example PVDES↑ and PVDES↓ both had
LCOEs of $116/MWh. While power systems at the Valley Site consider the costs
of the line going up to the Chajnator plateau, the derating factors and higher
costs in imported H2 increased costs at the high site. While cost-wise,
building on either side is reasonable, we cannot be certain that the
components considered can operate at the high altitude, as research in this
area is lacking. Testing on-site or in laboratory recreating the local
conditions will help to answer this.

Further, snowfall needs to be considered at the AtLAST Site, as snow layers
can reduce PV performance [53]. To account for this, higher OPEX due to snow
handling could be introduced. Moreover, one could find best locations for PV
arrays, that is sites with orography leading to less snow accumulation on the
panels.

###  4.3 Expanding the optimization: Life cycle analysis and energy
communities

GHG emissions as presented in this paper include direct emissions only. When
looking at the carbon footprint over the lifecycle of system components,
indirect emissions from raw material sourcing, production, transport and end-
of-life treatments should be considered as well, as RES incur indirect up- and
downstream GHG emissions. This analysis is the topic of an upcoming paper in
preparation by the authors. In this next work, we will include a life cycle
assessment within highRES-AtLAST. Using a multi-objective optimization on both
costs and lifecycle GHG emissions can generate a more profound answer to which
systems induces lowest CO2e-emissions.

The systems presented are built to serve a single telescope with an annual
demand of 7.7GWh. In proximity to AtLAST on the Chajnator plateau, eleven
other telescopes are operating and could be integrated into a joint power
system. This would not only reduce costs due to bigger scale, but also require
less area and materials per generated Wh. A Valley Site power system, powering
multiple telescopes on Chajnator, would only require one power line to the
plateau. One could moreover create an energy community with nearby localities,
supplying excess power to locals.

##  5 Conclusions

The authors found lowest LCOEs of $116/MWh for the telescope AtLAST with a
power system employing a combination of PV, batteries, imported and on-site
produced green H2 with CG storage and PEMFC, and backup diesel. Compared to a
business-as-usual setup with diesel generators only, systems based on PV
present 31-42% lower costs of power generation and more robustness regarding
cost sensitivities. They save 40-100% in direct GHG emissions compared to the
BAU scenario. Systems running on 100% RES with hybrid energy storage have 6-9%
higher costs than those with additional diesel as backup. Setups where PV was
used with only one main energy storage option, either batteries or H2, result
in LCOE jumps of 1-12%.

Synergies by building a centralized power system for the multitude of
telescopes on the Chajnantor plateau could lead to lower costs, e.g. in
maintenance and cabling. Further research is needed to conclude whether the
systems presented here can be employed at the altitudes discussed and what
indirect emissions they entail. The open-source system design optimization of
this work can also be applied for other (remote) facilities and thus serve as
a lighthouse to astronomical observatories and off-grid stationary
applications that would like to shift towards more sustainable power supply.

## Credits

Conceptualization, I.V., G.V.V., M.Z., and S.S.; methodology, I.V. and G.V.V.;
software, G.V.V.; investigation, I.V. and G.V.V.; data curation, I.V. and
G.V.V.; writing—original draft preparation, I.V.; writing—review and editing,
G.V.V., M.Z. and S.S.; visualization, I.V.; supervision, M.Z. and S.S.;
project administration, S.S.; funding acquisition, S.S. All authors have read
and agreed to the published version of the manuscript.

## Funding

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No. 951815.

## Data availability

The highRES-AtLAST model is published here: https://github.com/highRES-
model/highRES-AtLAST. Publicly available hourly solar irradiation time series
from ERA5 used in this study can be found here: https://www.ecmwf.int/en/
forecasts/datasets/reanalysis-datasets/era5. Restrictions apply to the
availability of the demand dataset, which was upscaled from a dataset provided
by APEX, not publicly available. A dummy demand dataset to obtain similar
results to the ones presented in this work is openly available within the
highRES-AtLAST model.

## Acknowledgements

We want to thank APEX for their kind collaboration. Special thanks goes to our
astronomy colleagues Claudia Cicone, Carlos De Breuck and Tony Mroczkowski for
the cooperation in this interdisciplinary project.

## Appendix

Table A.1: CAPEX and OPEX estimations in 2030, real2022-US$ values. Cost component | CAPEX | Unit | Fixed OPEX | Unit/year | Variable OPEX | Unit
---|---|---|---|---|---|---
Monofacial monocrys- talline PV | 523 [28, 29] | $/kWp | 1% [29] |  % of CAPEX | – | –
CSP Solar Field – PT / ST | 209/148[30] | $/m2 | 57.35 [30] | $/kWh | 3.48/3.04 [30] | $/MWh
Diesel generator | 495 [32] | $/kW | – | – |  23.5 (10 [32] +0.5$/l for ship- ping (assumed))
Li-Ion batteries | 262 [33, 34] | $/kWh | 0.43% [33] |  % of CAPEX | 0.5664 [33]
Electrolyzer - Alkaline / PEM | 434/483 [33] | $/kWe | 14.1[33] | $/kWe | 0.9394 [33] | $/MWhe
CG H2 storage | 598 [35] |  $/kg H2 stored | 2.1% [33] |  % of CAPEX | – | –
PEMFC | 1,581 [36] | $/kWe | 14.1 [33] | $/kWe | 0.9394 [34] | $/MWhe
Subterranean power lines, 24kV | 23,599 [37] | $/km |  0.5% (assumed) |  % of CAPEX | – | –
Table A.2: Results of base case cost optimization. Scenario | Unit | BAU | PVD↑ | PVD↓ | PVDES↑ | PVDES↓ | RES↑ | RES↓ | CSP
---|---|---|---|---|---|---|---|---|---
Annualized total |  |  |  |  |  |  |  |  |
electricity system cost | k$ | 1,544.7 | 1,059.9 | 1,112.8 | 894.9 | 893.3 | 946.1 | 978.0 | 2,921.6
Capacity |  |  |  |  |  |  |  |  |
Diesel | MW | 2.40 | 2.34 | 1.58 | 0.58 | 0.43 | – | – | –
PV | – | 2.27 | 2.38 | 4.20 | 4.76 | 4.75 | 5.44
Alkaline electrolyzer | MWe | – | – | 0.18 | 0.34 | 0.22 | 0.59
PEM electrolyzer | 0.68 | 0.39 | 1.01 | 0.91
CG storage | kg H2 | 263.1 | 159.8 | 1,048.7 | 1,399.3
PEMFC | MWe | 0.22 | 0.16 | 0.54 | 0.49
Li-ion battery | MWh | 15.1 | 14.2 | 15.2 | 12.4
CSP solar field | MW | – | – | – | – | 12.2
CSP ST | 1.14
Generation | annual |  |  |  |  |  |  |  |
Diesel | MWh | 7,697.6 | 4,523.5 | 4,594.9 | 540.8 | 375.2 | – | – | –
PV | – | 3,174.2 | 3,203.4 | 8,404.0 | 8,544.4 | 9,217.7 | 9,627.1
Thereof curtailed | 2,223.9 | 2,183.5 | 1,570.7 | 2,254.4 | 2,065.7 | 2,700.4
PEMFC | – | – | 735.2 | 534.9 | 1,204.7 | 1,201.9
Li-ion battery | 2,993.7 | 3,396.4 | 3,015.9 | 3,052.4
CSP solar field | MWhth | – | – | – | – | 20,668.6
Thereof curtailed | 2,815.0
CSP ST | MWh | 7,798.1

## References

  * [1] A. R. H. Stevens, S. Bellstedt, P. J. Elahi, and M. T. Murphy, “The imperative to reduce carbon emissions in astronomy,” Nature Astronomy, vol. 4, pp. 843–851, Sept. 2020.
  * [2] J. Knödlseder, S. Brau-Nogué, M. Coriat, P. Garnier, A. Hughes, P. Martin, and L. Tibaldo, “Estimate of the carbon footprint of astronomical research infrastructures,” Nature Astronomy, Mar. 2022.
  * [3] European Commission. Statistical Office of the European Union, “Energy, transport and environment statistics: 2020 edition,” _Energy, transport and environment statistics_ , 2020, Publications Office. URL: https://data.europa.eu/doi/10.2785/522192. Accessed: May 31, 2023.
  * [4] A. Acohido and C. Cavedoni, “Gemini Installs Record-Breaking Rooftop PV Solar Panel System | Gemini Observatory,” Oct. 2017.
  * [5] T. de Zeeuw, “Reaching New Heights in Astronomy - ESO Long Term Perspectives,” The Messenger, ESO, vol. 166, p. 26, 2016.
  * [6] Francisco Rodríguez, “ESO’s Paranal Observatory starts receiving energy from the largest solar plant in Chile dedicated to astronomy,” July 2022\.
  * [7] P. D. Klaassen, T. K. Mroczkowski, C. Cicone, E. Hatziminaoglou, S. Sartori, C. D. Breuck, S. Bryan, S. R. Dicker, C. Duran, C. Groppi, H. Kaercher, R. Kawabe, K. Kohno, and J. Geach, “The Atacama Large Aperture Submillimeter Telescope (AtLAST),” in Ground-based and Airborne Telescopes VIII, vol. 11445, pp. 544–563, SPIE, Dec. 2020.
  * [8] J. Ramasawmy, P. D. Klaassen, C. Cicone, T. K. Mroczkowski, C.-C. Chen, T. Cornish, E. da Cunha, E. Hatziminaoglou, D. Johnstone, D. Liu, Y. Perrott, T. Stanke, and S. Wedemeyer, “The Atacama Large Aperture Submillimetre Telescope: key science drivers,” in Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy XI (J. Zmuidzinas and J.-R. Gao, eds.), (Montréal, Canada), p. 9, SPIE, Aug. 2022.
  * [9] N. Endo, E. Shimoda, K. Goshome, T. Yamane, T. Nozu, and T. Maeda, “Simulation of design and operation of hydrogen energy utilization system for a zero emission building,” International Journal of Hydrogen Energy, vol. 44, pp. 7118–7124, Mar. 2019.
  * [10] C. Ghenai, T. Salameh, and A. Merabet, “Technico-economic analysis of off grid solar PV/Fuel cell energy system for residential community in desert region,” International Journal of Hydrogen Energy, vol. 45, pp. 11460–11470, Apr. 2020.
  * [11] K. Gebrehiwot, M. A. H. Mondal, C. Ringler, and A. G. Gebremeskel, “Optimization and cost-benefit assessment of hybrid power systems for off-grid rural electrification in Ethiopia,” Energy, vol. 177, pp. 234–246, June 2019.
  * [12] F. Dawood, G. Shafiullah, and M. Anda, “Stand-Alone Microgrid with 100% Renewable Energy: A Case Study with Hybrid Solar PV-Battery-Hydrogen,” Sustainability, vol. 12, p. 2047, Mar. 2020\.
  * [13] J. Price and M. Zeyringer, “highRES-Europe: The high spatial and temporal Resolution Electricity System model for Europe,” SoftwareX, vol. 17, p. 101003, Jan. 2022.
  * [14] M. Zeyringer, J. Price, B. Fais, P.-H. Li, and E. Sharp, “Designing low-carbon power systems for Great Britain in 2050 that are robust to the spatiotemporal and inter-annual variability of weather,” Nature Energy, vol. 3, pp. 395–403, May 2018.
  * [15] International Monetary Fund, “World Economic Outlook (October 2022) - Inflation rate, average consumer prices,” no date.
  * [16] R. Rondanelli, A. Molina, and M. Falvey, “The Atacama Surface Solar Maximum,” Bulletin of the American Meteorological Society, vol. 96, pp. 405–418, Mar. 2015.
  * [17] H. Hersbach, B. Bell, P. Berrisford, S. Hirahara, A. Horányi, J. Muñoz-Sabater, J. Nicolas, C. Peubey, R. Radu, D. Schepers, A. Simmons, C. Soci, S. Abdalla, X. Abellan, G. Balsamo, P. Bechtold, G. Biavati, J. Bidlot, M. Bonavita, G. De Chiara, P. Dahlgren, D. Dee, M. Diamantakis, R. Dragani, J. Flemming, R. Forbes, M. Fuentes, A. Geer, L. Haimberger, S. Healy, R. J. Hogan, E. Hólm, M. Janisková, S. Keeley, P. Laloyaux, P. Lopez, C. Lupu, G. Radnoti, P. de Rosnay, I. Rozum, F. Vamborg, S. Villaume, and J.-N. Thépaut, “The ERA5 global reanalysis,” Quarterly Journal of the Royal Meteorological Society, vol. 146, pp. 1999–2049, July 2020.  Publisher: John Wiley & Sons, Ltd.
  * [18] L. Ramirez Camargo, J. Valdes, Y. Masip Macia, and W. Dorner, “Assessment of on-site steady electricity generation from hybrid renewable energy systems in Chile,” Applied Energy, vol. 250, pp. 1548–1558, Sept. 2019.
  * [19] F. Hofmann, J. Hampp, F. Neumann, T. Brown, and J. Hörsch, “atlite: A Lightweight Python Package for Calculating Renewable Power Potentials and Time Series,” Journal of Open Source Software, vol. 6, p. 3294, June 2021.
  * [20] R. C. Muñoz, M. J. Falvey, M. Arancibia, V. I. Astudillo, J. Elgueta, M. Ibarra, C. Santana, and C. Vásquez, “Wind Energy Exploration over the Atacama Desert: A Numerical Model–Guided Observational Program,” Bulletin of the American Meteorological Society, vol. 99, pp. 2079–2092, Oct. 2018.
  * [21] D. Chattopadhyay, M. Bankuti, M. D. Bazilian, F. de Sisternes, S. Oguah, and M. Sanchez, “Capacity Planning Model with CSP and Battery,” in 2018 IEEE Power & Energy Society General Meeting (PESGM), (Portland, OR), pp. 1–5, IEEE, Aug. 2018.
  * [22] M. S. Răboacă, G. Badea, A. Enache, C. Filote, G. Răsoi, M. Rata, A. Lavric, and R.-A. Felseghi, “Concentrating Solar Power Technologies,” Energies, vol. 12, p. 1048, Mar. 2019.
  * [23] F. Schöniger, R. Thonig, G. Resch, and J. Lilliestam, “Making the sun shine at night: comparing the cost of dispatchable concentrating solar power and photovoltaics with storage,” Energy Sources, Part B: Economics, Planning, and Policy, vol. 16, pp. 55–74, Jan. 2021.
  * [24] A. A. Kebede, T. Coosemans, M. Messagie, T. Jemal, H. A. Behabtu, J. Van Mierlo, and M. Berecibar, “Techno-economic analysis of lithium-ion and lead-acid batteries in stationary energy storage application,” Journal of Energy Storage, vol. 40, p. 102748, Aug. 2021.
  * [25] R. Yudhistira, D. Khatiwada, and F. Sanchez, “A comparative life cycle assessment of lithium-ion and lead-acid batteries for grid energy storage,” Journal of Cleaner Production, vol. 358, p. 131999, July 2022.
  * [26] T. Egeland-Eriksen, A. Hajizadeh, and S. Sartori, “Hydrogen-based systems for integration of renewable energy in power systems: Achievements and perspectives,” International Journal of Hydrogen Energy, vol. 46, pp. 31963–31983, Sept. 2021.
  * [27] J. W. Pratt, J. Brouwer, and G. S. Samuelsen, “Performance of Proton Exchange Membrane Fuel Cell at High-Altitude Conditions,” Journal of Propulsion and Power, vol. 23, pp. 437–444, Mar. 2007.
  * [28] NAMA Facility, “Índice de Precios de Sistemas Fotovoltaicos (FV) conectados a la red de distribución comercializados en Chile,” tech. rep., Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH: Sede de la Sociedad Bonn z Eschborn, Nov. 2020.
  * [29] E. Vartiainen, G. Masson, C. Breyer, D. Moser, and E. Román Medina, “Impact of weighted average cost of capital, capital expenditure, and other parameters on future utility-scale PV levelised cost of electricity,” Progress in photovoltaics: research and applications, vol. 28, no. 6, pp. 439–453, 2020.  Publisher: Wiley Online Library.
  * [30] C. S. Turchi, M. Boyd, D. Kesseli, P. Kurup, M. S. Mehos, T. W. Neises, P. Sharan, M. J. Wagner, and T. Wendelin, “CSP Systems Analysis - Final Project Report,” Tech. Rep. NREL/TP–5500-72856, 1513197, May 2019\.
  * [31] S. Dieckmann, J. Dersch, S. Giuliano, M. Puppe, E. Lüpfert, K. Hennecke, R. Pitz-Paal, M. Taylor, and P. Ralon, “LCOE reduction potential of parabolic trough and solar tower CSP technology until 2025,” (Abu Dhabi, United Arab Emirates), p. 160004, 2017.
  * [32] Comisión Nacional de Energía, “Informe de costos de tecnologías de generación,” informe Anual, Comisión Nacional de Energía, Santiago de Chile, Mar. 2020.
  * [33] W. Cole, A. W. Frazier, and C. Augustine, “Cost Projections for Utility- Scale Battery Storage: 2021 Update,” Technical Report NREL/TP-6A20-79236, National Renewable Energy Laboratory, Golden, CO, 2021.
  * [34] K. Mongird, V. Viswanathan, J. Alam, C. Vartanian, and V. Sprenkle, “2020 Grid Energy Storage Technology Cost and Performance Assessment,” tech. rep., Tech. Rep., Pacific Northwest National Laboratory, US Department of Energy, 2020.
  * [35] L. Sens, U. Neuling, K. Wilbrand, and M. Kaltschmitt, “Conditioned hydrogen for a green hydrogen supply for heavy duty-vehicles in 2030 and 2050 – A techno-economic well-to-tank assessment of various supply chains,” International Journal of Hydrogen Energy, p. S0360319922031275, Aug. 2022.
  * [36] V. Cigolotti, M. Genovese, and P. Fragiacomo, “Comprehensive Review on Fuel Cell Technology for Stationary Applications as Sustainable and Efficient Poly-Generation Energy Systems,” Energies, vol. 14, p. 4963, Aug. 2021.
  * [37] CYPE Ingenieros, S.A., “Precio en Chile de m de Línea subterránea de distribución de baja tensión directamente enterrada. Generador de precios de la construcción.,” no date.
  * [38] International Renewable Energy Agency, “Future of Solar Photovoltaic: Deployment, investment, technology, grid integration and socio-economic aspects (A Global Energy Transformation: paper),” tech. rep., Abu Dhabi, 2019.
  * [39] Hydrogen Council and McKinsey & Company, “Hydrogen Insights Report 2021,” tech. rep., Hydrogen Council, Brussels, Belgium, 2021.
  * [40] Clean Hydrogen Partnership, “Hydrogen Europe - Strategic Research & Innovation Agenda,” tech. rep., Saint-Gilles, Belgium, 2019.
  * [41] Rystad Energy, “Research inquiry about diesel / crude oil price in 2030. Private communication,” July 2022.
  * [42] International Renewable Energy Agency, “Global hydrogen trade to meet the 1.5°C climate goal: Part III – Green hydrogen cost and potential,” tech. rep., Abu Dhabi, 2022.
  * [43] N. Endo, S. Suzuki, K. Goshome, and T. Maeda, “Operation of a bench-scale TiFe-based alloy tank under mild conditions for low-cost stationary hydrogen storage,” International Journal of Hydrogen Energy, vol. 42, pp. 5246–5251, Feb. 2017.
  * [44] L. Cabello, “Aprobación ambiental del parque Pauna, de 671 MWp, de Statkraft Chile,” Mar. 2022.
  * [45] European Commission. Joint Research Centre., Cost development of low carbon energy technologies: scenario based cost trajectories to 2050, 2017 edition. LU: Publications Office, 2018.
  * [46] Ministerio de Energía, Gobierno de Chile, “Planificación Energética de Largo Plazo 2023 - 2027: Informe Preliminar,” tech. rep., Ministerio de Energía, Subsecretaría de Energía; División Políticas y Estudios Energéticos y Ambientales; Unidad de Planificación Energética y Nuevas Tecnologías, Sept. 2021.
  * [47] International Energy Agency, “The Future of Hydrogen,” tech. rep., International Energy Agency, Paris, 2019.
  * [48] S. M. Saba, M. Müller, M. Robinius, and D. Stolten, “The investment costs of electrolysis – A comparison of cost studies from the past 30 years,” International Journal of Hydrogen Energy, vol. 43, no. 3, pp. 1209–1223, 2018.
  * [49] Ministerio de Energía, Gobierno de Chile, “National green hydrogen strategy. Chile, a clean energy provider for a carbon neutral planet,” June 2021.
  * [50] B. C. Tashie-Lewis and S. G. Nnabuife, “Hydrogen Production, Distribution, Storage and Power Conversion in a Hydrogen Economy - A Technology Review,” Chemical Engineering Journal Advances, vol. 8, p. 100172, Nov. 2021.
  * [51] Statkraft, “Statkraft ingresa a evaluación ambiental el proyecto solar más grande de Chile,” no date.
  * [52] Empresa Nacional del Petróleo, “Inversionistas y Mercado - Relaciones Comerciales - Tabla de precios de paridad,” no date.
  * [53] M. Santhakumari and N. Sagar, “A review of the environmental factors degrading the performance of silicon wafer-based photovoltaic modules: Failure detection methods and essential mitigation techniques,” Renewable and Sustainable Energy Reviews, vol. 110, pp. 83–100, Aug. 2019.


350

# Quantifying French Document Complexity

###### Abstract.

Measuring a document’s complexity level is an open challenge, particularly
when one is working on a diverse corpus of documents rather than comparing
several documents on a similar topic or working on a language other than
English. In this paper, we define a methodology to measure the complexity of
French documents, using a new general and diversified corpus of texts, the
“French Canadian complexity level corpus”, and a wide range of metrics. We
compare different learning algorithms to this task and contrast their
performances and their observations on which characteristics of the texts are
more significant to their complexity. Our results show that our methodology
gives a general-purpose measurement of text complexity in French.

###### keywords:

Keywords: Document Complexity, Readability Metrics, French Corpus, Machine
Learning

Vincent Primpied,*, David Beauchemin, Richard Khoury
---
Department of Computer Science and Software Engineering, Université Laval

<EMAIL_ADDRESS>
##  1\. Introduction

The evaluation of text complexity is intuitive to human readers. Upon looking
at a new document, a reader can almost immediately tell if the text is easy or
difficult to read. However, this task is more difficult for reading
comprehension systems [tolochko2019determining]. While text complexity
evaluation has evolved greatly recently in particular thanks to the rise of
machine learning (ML) [franccois2012nlp], most applications focus on measuring
the complexity of a specific type of text, such as political discourse
[tolochko2019determining] or administrative documents [franccois2014amesure],
and mostly in English [franccois2012ai]. As a result, most work on text
complexity focuses on a specific characteristic of the language, such as the
use of commas or the variety of words, which correlates with complexity in a
specific context but does not generalize well for all contexts nor all
languages [franccois2012ai].

This paper explores the design and training of a general-purpose text
complexity measuring algorithm in French. To achieve this goal, we pick out a
variety of metrics that the literature identified as significant to complexity
and train multiple learning algorithms to combine them to represent the
general, non-application-specific complexity of a wide range of documents. We
focus specifically on French, using our own French document corpus and metrics
specialized to the French language. Our methodology can be applied to other
languages as well by using a different corpus and metrics.

The rest of this paper is structured as follows. We will begin with a brief
overview of the literature on document complexity in section 2, and list the
metrics commonly used for that task in section 3. Then we will present our
work on training a complexity evaluation algorithm in section 4, including
notably how we constructed our French corpus, the various ML algorithms we
trained, the results they obtained and the features they each picked out as
significant to complexity. Finally, we will make some concluding remarks in
section 5.

##  2\. Related Work

One text-complexity measuring system is the Coh-Metrix Common Core Text Ease
and Readability Assessor [jackson2016common], which computes metrics
classified in 5 specific aspects of readability in literacy instruction:
narrativity, syntactic simplicity, word concreteness, referential cohesion and
deep cohesion. Although domain-specific, many metrics overlap with more
generic work. The authors of [kyle2018measuring] add that when evaluating
syntactic complexity, subdividing into clausal and phrasal indices can be very
relevant in the evaluation for English learner, and in a more generic
evaluation. The authors of [tolochko2019determining] proposed an analysis for
a global complexity measure of political discourse. They conceptualize text
complexity as two dimensions: syntactic complexity and semantic complexity.
After testing, they conclude that the complexity rating has to be based on a
multi-dimensional analysis to be relevant. It is interesting to note that the
same happened in the German language.

These studies were all developed for the English language and are not
compatible with a French application. The two languages have major
differences, such as French being much richer in morphology
[de2019development]. As a result, the tools and even basic concepts used to
quantify English complexity cannot be applied to French documents.

One example of a French text complexity system is AMESURE. It aims to evaluate
the complexity of administrative documents and to give advice on how to reduce
it [franccois2014amesure]. It allows users to see exactly what caused a grade
by highlighting too long or too complex sentences. However, while it performs
well in its intended domain, the system has difficulty assessing complexity in
a context besides administrative texts. This is because a reader experiences
text complexity according to their expectations of the document. Therefore,
measuring it depends on the application domain [tolochko2019determining], and
a domain-specific complexity measure makes domain-specific assumptions and
design choices that limit its transferability. Moreover, AMESURE is based on a
support vector machine that uses ten metrics representing three aspects of
complexity, limiting its score.

##  3\. Metrics

One of the most intuitive concepts of text complexity is the variety of
vocabulary used, or lexical diversity: the more diverse the vocabulary present
in a document is, the more complex it is. Hence, one of the first metrics of
text complexity created is the Type-Token Ratio (TTR) [johnson1944studies,
templin1957certain], which simply measures the ratio of unique words (types)
to total words (tokens) found in a document. Unfortunately, this metric is
very sensitive to the length of the text: longer texts have more tokens and,
therefore, a lower TTR score regardless of their actual difficulty level.
Thus, subsequent authors have expressed the TTR differently to combat this
issue [covington2007mattr, mccarthy2010mtld]. The Mean Sequential TTR (MSTTR)
[johnson1944studies, templin1957certain] computes the TTR on each 50-word
window of the document and the result is the averages of these TTRs. The
Moving Average TTR [covington2007mattr] does the same on a 100-word rolling
window instead of sequential windows. Finally, the Measure of Textual
Diversity (MTLD) [mccarthy2010mtld] takes the opposite approach: it sets a
target TTR value (the authors recommend 0.720) and gradually increases the
size of the text window until the TTR of that window falls below that
threshold, at which point a counter is incremented and the window is reset.
The document’s MTLD value is the ratio of the number of words to the counter
value.

Name | Description | Reference
---|---|---
TTR | Type-Token Ratio | [johnson1944studies, templin1957certain]
MSTTR | Mean Sequential TTR (50-word sequences) | [johnson1944studies, templin1957certain]
MATTR | Moving Average TTR (100-word window) | [covington2007mattr]
MTLD | Mean document length per TTR value | [mccarthy2010mtld]
Table 1. Summary of the lexical diversity metrics

Some authors note that there is more to vocabulary complexity than diversity:
a document with a diverse but simple vocabulary will seem less complex than
one with a limited vocabulary of difficult or technical words. The simplest
way to measure this notion of complexity is to count the percentage of words
not found in a list of simple words [dale1948formula]. The authors of
[si2001statistical] take a very different approach and define a list of
unigrams of words corresponding to different complexity levels, then count
occurrences of these words in texts. While these were done in English,
equivalent word lists are available in French, such as the Gougenheim
fundamental dictionary [gougenheim1958dictionnaire] or the Lexique 3 list of
lemmas and their probability of occurrence [new2005manuel]. Finally, going by
the assumption that longer words are more difficult, several authors propose
simply counting the average length of words [flesch1948readability,
smith1961devereux].

Name |  Description | Reference
---|---|---
PA |  Percentage of words not in the Gougenheim dictionary | [gougenheim1958dictionnaire, dale1948formula]
Unigram |  Unigram of lemma sorted by Lexique3 frequency | [new2005manuel, si2001statistical]
NLM |  Mean length of words | [flesch1948readability, smith1961devereux]
Table 2. Vocabulary complexity metrics

Another aspect of text complexity is syntactic complexity, or the use of
simpler or more complex grammatical constructions. The simplest metric to
measure this complexity is the Mean Length of Sentences (MLS), and its
variations in the “length of the 90th percentile sentence” or the “percentage
of sentences longer than 30 words” [flesch1948readability]. More accurate
metrics require first splitting each sentence into clauses, determining if it
is dependent or not and computing the Mean Length of Clauses
[flesch1948readability] and the Dependent Clauses per Clause
($\frac{\text{DC}}{\text{C}}$). An even more accurate metric has been found to
be the T-unit, which is defined as the root clause of a sentence along with
all its subordinate clauses except for conjunctions [hunt1965grammatical]. A
complex T-Unit (CTU) is a type of T-Unit with more than one clause111For
example, the conjunctive sentence “Il fait beau et les nuages sont partis.”
(“The weather is nice and the clouds are gone.”) contains two T-units and two
clauses, while the explicative sentence “Il fait beau parce que les nuages
sont partis.” (“The weather is nice because the clouds are gone.”) contains
one complex T-unit but two clauses.. In contrast to simply measuring clauses,
using T-Units makes it possible to isolate complex grammatical constructs.

The metrics we can draw from this shortest grammatically relevant sentences
and other grammatical units are listed in Table 3.

Name |  Description | Reference | Name |  Description | Reference
---|---|---|---|---|---
MLS |  Mean Length of Sentences | [hunt1965grammatical] | $\frac{\text{C}}{\text{TU}}$ |  Clauses per T-Units | [bardovi1989attainment]
MLC |  Mean Length of Clauses | [hunt1965grammatical] | $\frac{\text{CP}}{\text{C}}$ |  Coordinate phrase per Clauses | [lu2010automatic]
$\frac{\text{DC}}{\text{C}}$ |  Dependent Clauses per Clauses | [hunt1965grammatical] | $\frac{\text{CP}}{\text{TU}}$ |  Coordinate phrase per T-Units | [lu2010automatic]
MLT |  Mean Length of T-units | [hunt1965grammatical] | NWS90 |  Length of the $90^{th}$ percentile sentence | [hunt1965grammatical]
$\frac{\text{TU}}{\text{S}}$ |  T-Unit per Sentences | [hunt1965grammatical] | PS30 |  Percentage of sentences longer than 30 words | [hunt1965grammatical]
$\frac{\text{CTU}}{\text{TU}}$ |  Complex T-Unit per T-Units | [casanave1994language, lu2010automatic] |  |  |
Table 3. Syntactic complexity metrics

Finally, some authors propose to quantify and measure a document’s
readability. The traditional way of doing this is with the popular Flesch-
Kincaid reading ease formula [flesch1948readability], and his adaptation in
French the Kandel and Moles formula [kandel1958application]. This readability
measurement is given by Equation 1 and combines the MLS and the Mean Number of
Syllables (MNS). Other authors noted that readability is inversely related to
the use of commas. The intuition is that commas are likely used to introduce
additional dependent clauses in a sentence and thus make the sentence more
complex and more difficult to read. The BINGUI value [franccois2011apports]
measures this effect. For our work, we extend this metric to include semi-
colons, colons and parentheses, and commas, which all have a similar effect on
readability. These readability metrics are summarized in Table 4.

$\displaystyle\text{KM}_{\text{score}}=207-1.015\times\text{MLS}-0.736\times\text{MNS}$
(1)

Name |  Description | Reference
---|---|---
FK ease |  Flesch-Kincaid reading ease | [flesch1948readability]
KM score |  Flesch reading ease, adapted for French by Kandels and Moles | [kandel1958application]
BINGUI |  Commas per sentence, which we extend to include semi-colon, colon and parenthesis | [franccois2011apports]
Table 4. Readability metrics

In addition to these complexity metrics, we apply the Biber tagger
[biber1988adverbial] to each document in order to measure a set of linguistic
features present in the text. We use the implementation available through the
`biberpy` library [sharoff21rs], which measures the features listed in Table
5. We normalize these values as the ratio of each count to the number of words
in the text.

Metric | Description | Metric | Description
---|---|---|---
pastVerbs | Past tense verb | otherSubord | Other clauses
presVerbs | Present tense verb | preposn | Adposition
placeAdverbials | Place adverbials | attrAdj | Attributive adjectives
timeAdverbials | Time adverbials | ADV | Adverb
1persProns | 1st person pronouns | conjuncts | Conjunct
2persProns | 2nd person pronouns | downtoners | Downtoners
3persProns | 3rd person pronouns | amplifiers | Amplifiers
impersProns | Impersonal pronouns | generalEmphatics | Emphatics
demonstrProns | Demonstrative pronouns | publicVerbs | Speech act (public) verbs
indefProns | Indefinite pronouns | privateVerbs | Intellectual state (private) verb
doAsProVerb | Do as pro-verb | suasiveVerbs | Suasive Verbs
whQuestions | WH- questions words | seemappear | Occurences of “seem” or “appear”
nominalizations | Nominalizations | possibModals | Possibility modals
Nouns | Nouns (without nomalizations) | necessModals | Necessity modals
beAsMain | Be as sentence’s root | predicModals | Prediction modals
WHclauses | WH relative clauses | contractions | Word contractions
piedPiping | Pied-piping constructions | thatDeletion | Omission of “that”
sncRelatives | Relative clauses | strandedPrep | Preposition strandingbonjour
causative | Causative adverbial clauses | syntNegn | Syntactic negation
conditional | Conditional clauses | analNegn | Analytic negation
Table 5. List of metrics taken from the Biber tagger.

##  4\. Experiments

###  4.1. Corpus

Our first challenge is that no corpus exists to study general text complexity
in French. Consequently, we decided to build our own. We identified eight main
groups of documents representing distinct points on the spectrum of writing
styles and difficulty levels: children’s stories, recipes, news articles,
Wikipedia articles, novels, dictations, insurance documents and legal texts.
Children’s stories and recipes are the two simplest forms of text we sampled:
the former because it deliberately uses simpler syntax and vocabulary. The
latter is because it is a set of clear and concise directions. Novels, news
articles and Wikipedia articles are of medium complexity, as they are intended
for a more knowledgeable public. Novels are a more complex form of our
previous stories category. At the same time, news articles and Wikipedia
articles are both texts intended to convey a lot of information to the reader,
with news articles geared more toward the general public and Wikipedia
articles towards individuals interested in an in-depth technical explanation.
Dictations are intermediate between these; the text is more complex than
stories and recipes but less so than novels and articles. It will also contain
some outlier features intended to test the limits of the students’ knowledge.
Finally, insurance and legal documents, two forms of text that use a very
technical vocabulary and complex syntactic structures, are the most difficult
forms of text we sampled. The three main tiers of complexity (children’s
stories and recipes/news, Wikipedia articles, novels and dictation/insurance
and legal documents) are clear; however, the ranking within each tier is based
on our intuition. This, however, will not affect the quality of our results in
a significant manner. Future work may create a more objective tier ranking
using focus groups or a set of independent labellers. Note that we selected a
maximum of texts in Canadian French instead of other French dialects whenever
possible. Our small assembled corpus, the “French Canadian complexity level
corpus”, is composed of 118 documents and is available on our GitHub
repository.

###  4.2. Results per Metric

The first step of our study is to explore how each metric correlates with
document complexity. To do this, we compute each metric for each document
independently and group metrics that behave similarly. This allows us to
discern three different groups of behaviours for the metrics.

The first group is for metrics that are highly correlated with complexity but
overlap in values between similar categories. These metrics are useful to get
a coarse evaluation of complexity but cannot differentiate between fine-
grained levels of complexity. Moreover, a metric in this group can
occasionally misevaluate one level of complexity but be correlated for the
other seven. For example, the MLS is only based on the length of sentences and
will classify a straightforward amendment with short sentences as easy as a
children’s story.

One typical example of metrics in this group is the Kandel and Moles formula,
which we present in Figure 4: there is a clear progression of values from the
simplest to most complex level of texts, but too much overlap between levels
to reliably classify a document in one category based on that information
alone. The other metrics in that group are Flesch-Kincaid reading ease,
nominalizations, MLS and $\frac{\text{DC}}{\text{C}}$ .

Figure 1. Box plot of the Kandel and Moles formula for each complexity level

Figure 2. Box plot of the BINGUI metric for each complexity level

Figure 3. Box plot of the $\frac{\text{CTU}}{\text{TU}}$ metric for each
complexity level

Figure 4. Box plot of a metric not applicable to French, the discoursePart

The second group are metrics that overlap in value for most levels of
complexity but give very different results for one level (or few levels).

The BINGUI metric shown in Figure 4 is a good example of such a metric: it
gives very different results for legal documents but reliably cannot tell the
other seven levels apart. The other metrics in that group are the MSTTR, TTR,
conjuncts, generalEmphatics, strandedPrep, synthNegn, possibModals, MTLD,
MATTR, contractions, $\frac{\text{CP}}{\text{TU}}$,
$\frac{\text{TU}}{\text{S}}$, wordLength, $\frac{\text{C}}{\text{TU}}$,
pastVerbs, $PS_{30}$ and MLT. Their usefulness is their ability to isolate one
level of complexity from the others.

The final group is for metrics that do not fit in the previous two: they
present a low correlation with text complexity and do not have a single level
that stands out from others.

The $\frac{\text{CTU}}{\text{TU}}$ metric shown in Figure 4 is a good
representative of metrics in this group, and the others are listed in Table 6.
While such a metric does not seem helpful on its own, combined with other
metrics, it may prove helpful.

Unigram | ADV | amplifiers | preposn | downtoners | presVerbs | Nouns
---|---|---|---|---|---|---
PA | piedPiping | NLM | conditionnal | demonstrProns | doAsProVerb |
WHclauses | 2persProns | timeAdverbials | sncRelatives | analNegn | causative |
predicModals | impersProns | $\frac{\text{CP}}{\text{C}}$ | indefProns | otherSubord | $NWS_{90}$ |
necessModals | suasiceVerbs | whQuestions | 1persProns | thatDeletion | placeAdverbial |
attrAdj | 3persProns | publicVerbs | seemappear | beAsMain | privateVerbs |
Table 6. List of metric with a low correlation with text complexity

The Biber tagger identifies complexity forms that have not been considered.
Unfortunately, some of the concepts measured by the tagger do not apply to
French. Theses are easily identifiable: Figure 4 shows an example.

###  4.3. Results with Machine Learning

Since no metric can precisely differentiate between all eight complexity
levels, the next stage of our research is to use ML to find a way to combine
them and gain that ability. We will use a readability metric as a baseline to
make a relevant comparison.

Furthermore, to train our ML models, each complexity level is labelled as a
number class from 0 to 7. Zero is the easiest class (Children’s stories), and
seven is the most difficult (law documents).

The classifier models we experimented with are a Decision Tree (DT), Random
Forest (RF), a Logistic Regression (LR), and a Naive Bayes (NB), from `scikit-
learn`. First, we perform a grid search with cross-validation to optimize each
of our ML algorithms. For the NB, we perform the grid search on the Laplace
smoothing value. For the DT, we alternate between Gini or entropy as a quality
metric and optimize the maximum depth. As for the RF, we use the same
parameters as the DT plus the number of estimators in the forest. Finally, for
the LR, we search over the regularization strength. As a baseline, we use an
implementation of the Kandel and Moles formula with thresholds
[franccois2012nlp, benoit1986revue]. These thresholds are associated with the
definition of the Kandel and Moles formula [kandel1958application], as the
Table 8 show. Complete code is available on our GitHub repository.

Score | Description of style
---|---
90-100 | Very easy
80-90 | Easy
70-80 | Fairly easy
60-70 | Standard
50-60 | Fairly difficult
30-50 | Difficult
0-30 | Very difficult
Table 7. Kandel and Moles threshold according to the authors
[kandel1958application]

Model | RMSE | Accuracy
---|---|---
Baseline | 1.28 | 0.38
Decision Tree (DT) | 1.43 | 0.65
Logistic Regression (LR) | 0.38 | 0.83
Multinomial Naive Bayes (NB) | 0.36 | 0.84
Random Forest (RF) | 0.21 | 0.92
Table 8. RMSE and Accuracy of the baseline and our four best ML configurations
(bold value are the best performance per metric)

We measure two metrics of the quality of the results. The first is accuracy,
or the proportion of documents classified at the correct complexity level. The
second is the Root-Mean-Square Error (RMSE), which is computed by assigning to
each complexity level a sequential integer value. The RMSE will thus compute
how far off a misclassified document is from its correct class. So, for
example, a classifier that misidentifies a children’s story as a recipe (the
next complexity level) will achieve a lower RMSE than one that classifies it
as a legal document (the most complex level). Table 8 show the RMSE and
accuracy score for our baseline and four best models configuration.

Unsurprisingly, the baseline achieves the lowest accuracy and second-worst
RMSE. Moreover, as indicated previously and illustrated in Figure 4, the
Kandel and Moles metric gives progressively lower scores to increasing
categories of text complexity, and the scores of consecutive categories
overlap a lot. As a result, the baseline has trouble pinpointing an exact
complexity level for a text.

The DT classifier achieves a better accuracy but worse RMSE than the baseline,
and performs worse on both points than the other ML algorithms. DT combines
the information of several metrics while the baseline is only one metric, so
its higher accuracy compared to the baseline is not surprising. However, its
main weaknesses are that it focuses on a subset of the features available, and
it applies the same features to sort between all complexity levels at the
highest levels of the tree. As our analysis in subsection 4.4 will show,
different features are indicative of different complexity levels, and almost
every feature can come into play. This makes DT unsuited for this task, and
explains its lower performance compared to the other algorithms.

The LR, NB and RF classifiers all achieve very strong accuracy and RMSE scores
in our experiments. The LR classifier is built as a set of eight individual
one-vs-rest regression equations, each one computing the score of the document
belonging at that complexity level. The NB classifier computes, for each
complexity level, the product of conditional probabilities of that complexity
given the observed features of the text. The equation yielding the highest
score dictates the predicted complexity level in both cases. The RF classifier
is composed of an ensemble of 1,244 independent DTs. Most importantly, both LR
and NB consider all features available, and both can weight these features
differently for each complexity level independently of the others. Each DT in
the RF uses a different random subset of features, but the ensemble will
likewise make use of all features. This gives these algorithms the flexibility
needed to outperform DT and the baseline. The NB classifier also benefits from
smoothing the probabilities of unseen events, which is an advantage given the
small size of our dataset and allows it to beat out the LR in the RMSE score.
The RF, on the other hand, benefits from merging the decisions of an ensemble
of individual trees, each trained to test their subset of features optimally.
This flexibility allows it to beat out the other ML algorithms and achieve the
best accuracy and RMSE.

It is interesting to look more closely at the use of the features made by ML
classifiers. With the LR, normalized metrics that have large positive or
negative coefficient values are the ones that have the most influence on the
score of a level. In contrast, those with values near zero have little to no
influence. Moreover, metrics whose coefficients vary significantly from one
equation to the other will have a greater ability to differentiate one
complexity level from another. In comparison, metrics whose coefficient values
remain constant from one equation to another have no discerning power. Using
this standard, we can discern many significant metrics that distinguish
between two or more levels of complexity. The TTR, for instance, has a very
high positive value for Wikipedia articles and a very high negative value for
legal documents, owing to the former’s linguistic variety and the latter’s
repetition of formally-defined terms. Conjuncts (the use of conjunctions) is
also a strong positive indicator of the dictations and a strong negative
indicator of recipes, showing the former class’ reliance on syntactic
complexity to challenge learners and the latter’s use of clear and simple
statements. There are also metrics that distinguish groups of levels. For
instance, contractions have a strong positive coefficient for children’s
stories and dictations, and a strong negative coefficient for Wikipedia
articles, novels, and legal documents. Some metrics are strong indicators for
only one complexity level and null for others. For example, this is the case
for the use of past-tense verbs (strong positive coefficient for insurance
contracts) or the use of adverbs (strong negative coefficient for legal
documents). Finally, a few metrics had near-zero coefficients in all LR
equations. This was notably the case for the use of second-person pronouns,
pied-piping, suasive verbs, time adverbs, or the use of seem/appear.

A similar observation holds true for the NB. Since our metric values are
normalized, a significant indicator for a complexity level metric will have a
high conditional probability. Moreover, if a metric is a significant indicator
of one complexity level but not of another, the conditional probability of the
first level, given that metric will be high and that of the second level, will
be low. By that standard, the most significant metrics according to NB include
the use of contractions, syntax negation and 3rd person pronouns. The first
metric is a high probability for children’s stories, recipes and dictations
but a low probability for novels and legal documents. The second is a high
probability for insurance and legal documents but a low probability for
Wikipedia articles. The third metric is a high probability for stories, news
articles and Wikipedia articles but a low probability for recipes and
insurance contracts. On the other hand, insignificant metrics that have
uniformly low probability values for all or almost all complexity levels
include pied-piping, suasive verbs, time adverbs and the use of seem/appear.
All of these also had coefficients near zero in all LR equations. Equally
interesting are metrics that have uniformly high probability values for most
complexity levels, the most striking of which are the FK ease and KM score.
These reading ease metrics are correlated with text complexity but overlap in
values for neighbouring complexity levels, as was shown in subsection 4.2.
This seems to keep the LR from using them, but does not hinder the NB.

Unlike the LR and NB, the DT and RF create one classifier for all complexity
levels. This means that we can weight the importance of each metric to the
classification, but we cannot determine which metric is significant for each
complexity level. We find the DT uses 15 metrics, while the RF unsurprisingly
uses all of them. More interestingly, when sorted by importance, the order of
metrics roughly agree, with more or less important metrics picked by the DT
weighted as more or less important by the RF as well. The most important
features agreed upon by both classifiers are the TTR and MTLD scores, the
$\frac{\text{CP}}{\text{C}}$ and $\frac{\text{DC}}{\text{C}}$ ratios, the use
of impersonal pronouns and nominalizations, and the KM score. Many of these
metrics are also rated as strong indicators of various complexity levels by
the LR and NB classifiers as well.

###  4.4. Assessing French Text Complexity

Overall, it seems the ML classifiers isolated, for each class of documents, a
set of metrics that describe what characteristics documents of that class
typically have or do not have. Since we selected the eight classes to
represent eight different levels of text complexity, these characteristics
should also match those that make documents easier or more difficult to read.
Following our discussion of subsection 4.3, we decided to focus on the LR
classifier. While it does not perform as well as the NB or RF in Table 8, its
performances are still good, and more importantly it is directly interpretable
from the coefficients of the features in each equation. Table 9 presents the
list of metrics that receive strong positive or negative coefficients, which
we define as an absolute value above 0.1, for each complexity level. Looking
at those results makes it possible to see features of text complexity emerge.

  * •

Legal documents, the most complex form of text in our study, represent a form
of syntactic complexity that uses long and complex sentence structures with
T-Units, dependent clauses, coordinate phrases, and commas. The vocabulary is
more formal, with low diversity and little use of contractions and adverbs.

  * •

Insurance contracts, the other most complex form of text, represent instead a
form of grammatical complexity, making use of modal verbs, long words, and
stranded prepositions.

  * •

Dictations show a form of vocabulary complexity, including nouns, adjectives,
contractions, and rare words, but simpler sentences with to be as the main
verb.

  * •

Novels, one of our intermediate levels of text complexity, represent a more
sophisticated prose, with prepositions, causative verbs, downtoners and
emphatics to modify other words, and a lack of coordinate phrases.

  * •

Wikipedia articles represent a more technical form of prose, with a high
vocabulary variety, longer words, and few contractions, but simple sentences
using verbs like to be and to do.

  * •

News articles are the simplest of our intermediate complexity levels,
characterized by vocabulary diversity and a heavy use of pronouns.

  * •

Recipes, the first of our two easiest levels of difficulty, are characterized
by short sentences that avoid any sophistication such as “wh-” subordinate
clauses, conjunctions, questions, and “that” deletion.

  * •

Children stories are the simplest level of text complexity in our study, and
are characterized by the use of simplifying acts such as that-deletions and
contractions, and the use of shorter words and short sentences.

Another interesting observation is that most of our metrics are useful in
modelling text complexity. Indeed, we had included several related metrics in
our study, which measure the same characteristics but with slight variations.
We expected to see some of them retained by the ML algorithms and others
ignored, but this turned out not to be the case. For example, we included four
variations of the TTR metric in Table 1, and all four were used to identify
some complexity levels. It indicates that the characteristics that make a text
easier or harder to read can be very pointed, and thus the differences between
these variations of metrics capture useful information.

Complexity |  Positive Coefficient |  Negative Coefficient
---|---|---
Legal document |  BINGUI, $\frac{\text{C}}{\text{S}}$, $\frac{\text{CP}}{\text{C}}$, $\frac{\text{CP}}{\text{TU}}$, $\frac{\text{DC}}{\text{C}}$, MLS, nominalizations, possibModals, $\frac{\text{TU}}{\text{S}}$ |  ADV, TTR, contractions, KM score, MSTTR, $PS_{30}$
Insurance document |  $\frac{\text{C}}{\text{TU}}$, conditional, indefProns, necessModals, NLM, pastVerbs, predicModals, strandedPrep, syntNegn, wordLength |  (none)
Dictation |  Nouns, attrAdj, beAsMain, conjuncts, contractions, NLM |  $\frac{\text{C}}{\text{TU}}$, indefProns, unigram
Novels |  1persProns, analNegn, causative, $\frac{\text{CTU}}{\text{TU}}$, downtoners, generalEmphatics, MLT, preposn, publicVerbs |  3persProns, contractions, $\frac{\text{CP}}{\text{C}}$, $\frac{\text{CP}}{\text{TU}}$
Wikipedia article |  TTR, beAsMain, $\frac{\text{CTU}}{\text{CU}}$, doAsProVerb, MSTTR, wordLength |  analNegn, causative, conditional, contractions, preposn, sncRelatives
News article |  1persProns, 3persProns, analNegn, demonstrProns, impersProns, MATTR, MTLD |  NLM
Recipe |  (none) |  WHclauses, attrAdj, beAsMain, conjuncts, MLT, nominalizations, pastVerbs, presVerbs, thatDeletion, whQuestions
Story |  3persProns, amplifiers, contractions, impersProns, placeAdverbials, privateVerbs, thatDeletion |  MLT, PA, wordLength
Table 9. Metrics with strong positive and negative coefficients (LR
classifier) for each text complexity level

###  4.5. Blind Tests

We decided to run a final test of our methodology. While our classifiers work
well to recognize the complexity levels of documents belonging to the
categories seen during training, we now want to see if they can also handle
documents of related complexity levels but of different categories or genres.
To this end, we picked six new categories of documents and collected 5 sample
texts for each. They are:

  * •

movies subtitles, documents consisting of short simple sentences akin to
children stories and recipes and which should thus rank at levels 0 or 1;

  * •

tweets, also simple texts but from a very different source than the previous;

  * •

introductory-level college course materials, informative documents aimed at
non-specialized learners. They should be a bit more complex than Wikipedia or
news articles, which are informative documents aimed at the general public, at
level 4;

  * •

philosophical essays, the most different type of documents from our original
corpus, but which we expect to be around complexity levels 4 or 5, meaning
they would be more complex than general-public-level technical documents but
less complex than specialized legal documents;

  * •

websites terms of services, which are equivalent to insurance contracts at
level 6;

  * •

laws from French-speaking European countries, equivalent to level 7.

We then classified each document independently into a complexity level using
the four classifiers we trained in subsection 4.3, and computed the average.
The resulting detected complexity levels are given in Table 10.

Category | Expected | DT | LR | NB | RF
---|---|---|---|---|---
Movies subtitles | 0-1 | 0.0 | 3.6 | 0.0 | 2.0
Tweets | 0-1 | 2.0 | 3.2 | 1.2 | 3.6
College materials | 4 | 4.8 | 4.8 | 3.8 | 4.5
Philosophical essays | 4-5 | 4.0 | 5.4 | 1.8 | 3.6
Terms of Service | 6 | 5.2 | 6.0 | 6.0 | 5.4
Foreign laws | 7 | 5.0 | 6.0 | 5.8 | 4.8
Correlation to expected levels |  | 0.90 | 0.98 | 0.90 | 0.82
Table 10. Predicted complexity levels of the blind test document set

The results show some interesting patterns. To begin, almost all algorithms
overestimate the complexity levels of subtitles and tweets. It may be because
the two least complex classes we trained with, recipes and news articles, were
marked by low vocabulary diversity, which is not the case for either of these
test classes. In fact, tweets, with their frequent use of spelling mistakes,
hashtags, usernames and pop-culture terms, have a very high vocabulary
diversity, which misleads most classifiers into ranking them near the middle
of our complexity levels. Likewise, the massive use of pronouns in subtitles,
which are transcripts of spoken discussions, is a typical trait of the news
article class and leads to overestimating their complexity by two classifiers.
On the other hand, the complexity of the foreign law class is very under-
estimated by all four classifiers. This is because French-Canadian laws, which
form our level-7 complexity (highest level), are very different from European
laws, which appear more akin to Canadian insurance contracts. For example,
Belgian laws are structured as short one-sentence articles, whereas Canadian
ones use complex compound articles with lists of cases. On the positive side,
however, the classifiers were good at pinpointing the complexity levels of
college course materials, philosophical essays (except for NB), and terms of
service documents.

More importantly, it can be seen that all four classifiers respect the order
of rankings of complexity levels. They all find the Tweets and subtitles to be
the simplest levels, the terms of service and foreign laws to be the most
complex levels, and the college materials and philosophical essays of
intermediate complexity between them. Moreover, the correlation coefficient
between our expected complexity levels and those produced by each of the
classifiers ranges between 0.82 and 0.98. They thus classify text complexity
in a manner coherent with our intuition, with the LR ranking complexity in an
order closest to ours. This observation confirms that our methodology picks
out document complexity and does not simply classify documents by type or
topic.

##  5\. Conclusion

In this paper, we studied the concept of written text complexity in French. To
this end, we built a corpus of 118 French documents covering 8 levels of
complexity ranging from children’s stories to insurance contracts, and
selected some 60 different popular metrics of text complexity. Our first
realization is that none of these metrics can, by itself, correctly model the
range and diversity of text complexity. Consequently, we applied a variety of
ML algorithms, and found that logistic regressions, Naive Bayes, and random
forests all can discover good combinations of metrics to predict text
complexity, with RF edging out the other two in performance while the LR is
most explainable. Taking advantage of that explainability, we were then able
to infer the set of features that define each complexity level. This in turn
made it possible to apply our algorithms successfully to new types of
documents and correctly measure and rank their complexity, with LR giving the
most human-like rankings. Our corpus, code and results are all available on
our GitHub repository.

Our proposed method is, to the best of our knowledge, the only general-purpose
algorithm to accurately measure a wide range of text complexity in French.
Moreover, the explainability of the method gives insight into the nature of
the French language as well as confidence in the predicted complexity level of
a document. Finally, while we did not experiment in that direction, we believe
our methodology can be applied to different languages as well, by creating a
corpus in the new language and possibly adding new metrics to model unique
aspects of that language’s grammar.

## Acknowledgements

This research was made possible thanks to the support of La Capitale
Assurances et Services Financiers and NSERC research grant RDCPJ 537198-18.