# Measuring IT Carbon Footprint: What is the Current Status Actually?

Tom Kennes  Skyworkz/SDIA, Amsterdam, The Netherlands

###### Abstract

Despite the new Corporate Sustainability Reporting Directive from the European
Union, which presses large enterprises to be more transparent about their GHG
emissions, and though large technology- or advisory firms might peddle
otherwise, there are plenty of challenges ahead when it comes to measuring GHG
emissions from IT activities in the first place. This paper categories those
challenges into 4 categories, and explains the current status, shortcomings
and potential future research directions. These categories are: measuring
software energy consumption, server overhead energy consumption, Energy Mix
and emissions from embodied carbon. Next to that, various non-profit and open-
source initiatives are introduced as well as a mathematical framework, based
on CPU consumption, that can act as a rule-of-thumb for quick and effortless
assessments.

##  1 Introduction

The latest report by the IPCC [8] is clear. Urgent and drastic action is
needed if we are to limit global warming to 1.5°C! The same holds true for the
Information Technology industry (IT; the collection of software, digital
applications, digital services and code running on servers). Even more so as
the world accelerates its adoption of digital services and businesses
increasingly tend to rely on software to drive their operations.

Notwithstanding, historically it has been difficult to put a number on the
total energy consumed and GHG emission by IT worldwide, and recent research
has been mostly dominated by figuring out appropriate assumptions and scope
determination ([4], [6], [43], [46]). This might explain why these discussions
have not had more attention, a part of the public Netflix debate, to properly
account for GHG emissions by IT. This debate, with the IEA and CarbonBrief
([44], [11], [35]) on one side, and the Shift Project on the other side ([17],
[21]), focuses on the environmental impact of streaming one hour of Netflix
videos. Some of the disagreement can perhaps be best explained by Koomey’s Law
([42], [40]), or the doubling of the energy efficiency of computers every 18
months leading to an increase in scientific awareness as to properly forecast
the impact of future energy efficiency gains ([2], [3], [41]). All in all, the
debate finely serves to illustrate some of the underlying issues.

Three years later and we still are not sure how accurate most of the top-down
estimates really are. Hence, it might be more effective for the overall
transition to focus on more accurate bottom-up measurements. Similar
conclusions follow from [51] and [47]. Top-down estimates might provide
information for policymakers and global strategists, but they lock out workers
that are responsible for and able to curb those same emissions. In the end,
climate change is everybody’s problem, so we should be involving everybody, as
is argued by [52] as well.

As such, perhaps the numbers are accurate on some levels, but without lower-
level trustworthy numbers it will remain difficult if not impossible to
incorporate this into lower-level decision making and stimulate lower-level
efforts. This is a problem, especially in areas where highly-skilled lower-
level workers may have much more impact than higher-level global strategy
management, such as IT.

For IT departments to lower their emissions, this implies empowering
developers and engineers by tools that allow them to perform measurements, as
well as stimulating management to incorporate this into their strategy.
Luckily but perhaps unknowingly, IT management often already has tools and
strategies for cost-management in place that lead to more sustainable IT
practices. Lower IT utilisation is simply more green as well as more
affordable. The resulting decrease in emissions, whether incidental or not, is
something praiseworthy. Especially if the exact number could also be reported
and audited.

However, almost every IT department is either not yet measuring its GHG
emissions, or putting out laudable goals with questionable details. And for
good reason, it is simply not straightforward to measure the GHG emissions of
software, code and IT infrastructure. Despite the multitude of engineering
efforts to bridge this gap, there still exist significant problems that need
tackling.

This paper focuses on those gaps. Coming from a practical point of view,
measuring GHG emissions due to IT can be divided into 4 elements. These
elements are:

  * •

Software Energy Consumption ($E_{s}$)

  * •

Server Overhead Energy Consumption ($E_{o}$)

  * •

Momentaneous Energy Mix ($I$)

  * •

Embodied Carbon Emissions ($M$)

For each of those elements, this paper details the current status and
possibilities, and proposes various research questions to advance our overall
capability. The goal is to better align open-source initiatives, practitioners
and academic research in the future. It is believed that, tackling these
problems and improving our overall capability to measure the energy
consumption and GHG emissions of software, will aid significantly in reducing
the carbon footprint of IT.

Finally, note that these 4 elements need to be combined in order to calculate
the actual carbon emissions. Logically, these can be separated into, on the
one hand, operational carbon emissions, the energy consumed by the software
and the overhead multiplied by the carbon emissions related to producing that
energy in the first place, and, on the other hand, the embedded carbon
emissions. In short:

$TotCarbon=\big{(}(E_{s}+E_{o})*I\big{)}+M$ (1)

The observant reader will recognize the formula as the Green Software
Foundation (GSF) Software Carbon Intensity (SCI) specification ([23]) at the
moment of writing. The difference is the separation of Software and overhead,
as well as the absence of what the SCI refers to as ”the functional unit” (R):

$\frac{TotCarbon}{R}=SCI=\big{(}(E*I\big{)}+M)perR$ (2)

Although the GSF SCI is a good effort coming from renowned practitioners, by
opting for flexibility, it does not dig deeper into the underlying problems.
Hence, adoption remains difficult. At the same time, the SCI is getting
traction and becoming a starting point for budding sustainable software
engineers. It therefore remains a great tool for awareness creation and
education, and a recommended read.

##  2 Software Energy Consumption

The most reliable method of measuring energy consumption by appliances remains
using physical multimeters, whereas obtaining accurate empirical power
measurements through software remains a challenge. According to a recent
academic survey, focusing on the energy consumption of machine learning models
[20] there are roughly 2 ways to do so:

  * •

Power Monitoring Counters (PMC)-based models

  * •

Simulation models based on computer-architecture

Note that out of the 19 analysed methods, only 6 are not based on PMC.

In this paper, we offer a third simplified way in order to both bring together
our knowledge to more expressive mathematical formulas as well as to
communicate and lobby the audience where to direct focus.

###  2.1 PMC-based measurements

Examples of PMCs commonly reflect statistics related to CPU instructions,
memory- or disk-operations and resource stalls. In the past, they were mostly
relevant for hardware development and testing, like for example chip
performance and chip heat dissipation. As such, there has been ample research
on the topic of PMCs, see for example [22], including the development of
various models, most notably RAPL (Running Average Power Limit) [49]. Another
popular model is IPMI, but it is often considered to be insufficiently
accurate ([36]).

RAPL has been developed by Intel, and is said to have reasonable accuracy
([38], [25], [26], [14]), although there has also been research advocating
more cautiousness [50]. All in all, the true accuracy remains a bit difficult
to determine and seems to be highly affected by different computer
architectures, BIOS settings and further performance configurations, which
makes adoption difficult. These are merely observations from practice as there
has been a lack of research on the effect of these factors. There are some
efforts to come to a generalised model based on RAPL data, which are discussed
in the next subsection.

As an important sidenote, RAPL has a known vulnerability and corresponding
mitigations ([31]).

###  2.2 Existing Data and Tools

Accurate or not, at the moment of writing there already exist numerous energy
measurement tools and applications, that are either directly powered by RAPL
(see, among others, [37], [30], [48], [60], [63], [45], [39], [66], [32],
[61]) or using forecasts based on earlier collected measurements of (somewhat)
identical machines (see, among others, [1], [53], [12], [62]). These earlier
collected measurements are generally either obtained via SPEC (Standard
Performance Evaluation Corporation) [57] or collected through RAPL in fully
controlled environments ([13], [12]), or even both.

SPEC has been executing performance benchmark assessments for over 30 years,
mostly through direct power measurements. Although there are indications that
these assessments are reasonably rigorous ([28], [57], [34]), it is known that
replication of these measurements is a challenge, in particular due to
specific performance-enhancing BIOS settings. Perhaps this explains the lack
of audits of the SPEC benchmark assessments. Next to that, it is not clear to
what extent the assessments can be generalised, nor whether they are nowadays
still relevant to assess the performance and energy consumption of modern
cloud environments, which have changed significantly over the last decade. For
example, the latest benchmark (the SPEC Cloud IaaS assessment of 2018 [58])
attempts to measure generic Cloud IaaS performance using two specific
exercises:

  * •

Stress Testing Apache Cassandra, using the Yahoo Cloud Serving Benchmark (see
also [9])

  * •

K-means clustering using Apache Hadoop (see also: [33])

It should be clear that these Cloud IaaS assessments only apply to a very
narrow context and thus do not allow drawing conclusions about overall Cloud
or Cloud IaaS performance. Cloud consists of an extremely vast set of services
and configurations that are continuously evolving at many different levels.
Hence, it is not useful to assess Cloud as a single entity without assessing
its underlying services and components.

All in all, the effort by SPEC to develop these benchmarks and come up with
measurements that can in turn be used to assess the energy consumption during
runtime is praiseworthy. It’s a great initiative to come to a centralised
database, yet the need for more rigorous measurements, transparency,
replicability and academic research should be clear.

###  2.3 Potential Improvements and Research Questions

Given the analysis in the previous section, there are a couple of research
questions and directions:

  * •

Continue the RAPL assessments and development. From a practical point of view,
the requirements for root access on a machine significantly complicates
adoption. If this could be alleviated, this should be more straightforward.

  * •

Improve our understanding of BIOS and performance configuration settings and
their actual impact on performance vs energy consumption.

  * •

Further integration of RAPL energy assessments with applications, software and
code

  * •

Continue the SPEC initiative, but in a more transparent and rigorous manner.
The focus should lie as much as possible on lower level components, rather
than the specific software that is actually causing the energy consumption.
The direct relationship between CPU and energy consumption is much more
interesting than the indirect relationship between software and energy
consumption. Besides, CPU and memory usage is easier tracked in most
environments. Yet, the latter might be helpful when deciding on IT
architecture and costs.

###  2.4 SDIA DEF Model

It is believed by the SDIA that it is beneficial to have a rule of thumb for
quick decision making and more rough estimations in environments where direct
measuring is simply not possible yet (Cloud). This is why we introduce the
SDIA DEF Model (Sustainable Digital Infrastructure Alliance Digital
Environmental Footprint), which can also be used as a general mathematical
framework to embed future research and development. The rationale behind the
model involves bringing down somewhat complicated energy models into metrics
that are easier to observe to some extent. In other words, CPU usage.

####  2.4.1 Generic Formulations

First of all, we split out a server into 4 of its core functionalities: CPU,
memory, storage or IO and networking.

Given that the total energy consumed by a server can be written as the sum of
the energy consumed by its components, as well as a baseline energy
consumption representing idle mode, we can write:

$E_{tot}=E_{cpu}+E_{mem}+E_{IO}+E_{net}+\beta_{idle}$ (3)

Now, we can generally not measure the energy consumption for any of these
components directly. However, we can split them into a factor we can measure,
component usage ($U$), and a factor that translates that observable into
energy ($f$). Note that we thus gather the underlying complexity of converting
usage into energy into the second factor ($f$).

Any improvements in our ability to accurately measure software energy
consumption would be by improving that second factor ($f$).

The total energy used by a server is thus defined as:

$E_{tot}=U_{cpu}f_{cpu}(U_{cpu})+U_{mem}f_{mem}(U_{mem})+\\\
U_{IO}f_{IO}(U_{IO})+U_{net}f_{net}(U_{net})+\beta_{idle}$ (4)

Whereas:

  * •

$E_{tot}$: The total energy used by a server

  * •

$PUE$: Power Usage Effectiveness. The overhead, mostly due to cooling.

  * •

$U_{cpu}$: Current CPU usage, measured in number of CPUs

  * •

$f_{cpu}(U_{cpu})$: Function of the CPU energy consumption per CPU usage in
number of CPUs

  * •

$U_{mem}$: Current memory usage, measured in bits

  * •

$f_{mem}(U_{mem})$: Function of the memory energy consumption per memory usage
in bits

  * •

$U_{IO}$: Current storage usage, measured in bits

  * •

$f_{IO}(U_{IO})$: Function of the storage energy consumption per storage usage
in bits

  * •

$U_{net}$: Current network usage, measured in bits

  * •

$f_{net}(U_{net})$ : Function of the network energy consumption per network
usage in bits

  * •

$\beta_{idle}$: Energy consumption of the server when idling

Note that the functions denoted by $f_{x}(U_{x})$ represent the energy
allocation per unit of usage. Its unit is thus $E/U_{x}$.

Also note that these are not necessarily linear. The functional depiction does
not assume anything about the nature of the relationship between component
usage and energy consumption. There might be other factors that should also be
incorporated, such as server type or server age.

####  2.4.2 Assumptions

The Thermal Design Power (TDP), or the maximum amount of heat a CPU is
designed to be able to dissipate without overheating, and is thus often used
as a proxy for the energy consumed by a CPU when it runs on max performance,
see for example [59]. Nonetheless, we admit that TDP may not be the best
metric to evaluate power consumption. Chips can easily consume more power than
their TDP specification for a short period of time. However, TDP can be a
reasonable estimation of the average power consumption when a CPU runs regular
workloads at its base clock.

Assumption 1 We can use the TDP as a proxy for energy consumption by a single
CPU core when fully using the CPU

Because the energy consumption of the other components generally is more
difficult to predict, we relate those directly to the energy consumed by the
CPU in the form of fixed allocation factors ($\mathbf{\alpha}$).

Assumption 2 For server running on full performance:

  * •

$\alpha_{CPU}$ of the energy consumed by the server is due to CPU

  * •

$\alpha_{mem}$ of the energy consumed by the server is due to memory

  * •

$\alpha_{IO}$ of the energy consumed by the server is due to storage

  * •

$\alpha_{net}$ of the energy consumed by the server is due to networking

  * •

All energy of a server can be allocated to these 4 components:
$\alpha_{CPU}+\alpha_{mem}+\alpha_{IO}+\alpha_{net}=1$

More often than not, however, we are not running a server on full performance.
In order to estimate the energy consumed in those scenarios, the SDIA proposes
to use linearity.

Assumption 3 The energy consumed by a server increases linearly relative to
the increase in usage of any of its components.

Assumption 4 Idling consumption is expected to be zero. After all, if none of
the components of the system are using energy, the system itself should
logically also not be using energy.

####  2.4.3 Derivation

Let $E_{tot-max}$ denote the total energy consumed by a server when running on
full performance.

And, using the first assumption:

$E_{cpu-max}=\\\ U_{cpu-max}f_{cpu}(U_{cpu-max})=TDP*N_{CPU}$ (5)

Whereas:

  * •

$N_{CPU}$ denotes the number of CPUs

Then, using the second assumption:

$E_{tot-max}=E_{cpu-max}/\alpha_{CPU}=[TDP*N_{CPU}]/\alpha_{CPU}$ (6)

Substituting $E_{cpu-max}$ by $U_{cpu-max}f_{cpu}(U_{cpu})$ , using our third
assumption, and solving for $f_{cpu}(U_{cpu})$, we get:

$f_{cpu}(U_{cpu})=(TDP*N_{CPU})/U_{cpu-max}$ (7)

And finally for the energy related to the CPU:

$E_{cpu}=U_{cpu}f_{cpu}(U_{cpu})=\\\ (U_{cpu}/U_{cpu-max})([TDP*N_{CPU}])$ (8)

For the other components, we get:

$E_{tot-max}=E_{mem-max}/\alpha_{mem}=\\\ [U_{mem-max}f_{mem}(U_{mem-
max})]/\alpha_{mem}$ (9)

Plugging in for $E_{tot-max}$, and solving for $U_{mem}f_{mem}(U_{mem})$:

$\alpha_{mem}[[TDP*NCPU]/\alpha_{CPU}]=\\\ U_{mem}f_{mem}(U_{mem})$ (10)

Or:

$(\alpha_{mem}/\alpha_{CPU})[TDP*N_{CPU}]=\\\ U_{mem-max}f_{mem}(U_{mem-max})$
(11)

Solving for $f_{mem}(U_{mem})$, and since $f_{mem}(U_{mem})$ does not actually
depend on $U_{mem}$:

$f_{mem}(U_{mem})=\\\ (\alpha_{mem}/\alpha_{CPU})[TDP*N_{CPU}]/U_{mem-max}$
(12)

Meaning we end up with the total energy related to memory:

$E_{mem}=\\\ (U_{mem}/U_{mem-max})(\alpha_{mem}/\alpha_{CPU})[TDP*N_{CPU}]$
(13)

The energy related to the other components can then be found in a similar
fashion, due to symmetry.

Given that the allocation vector $\alpha$ is a constant, the energy functions
$f_{x}(U_{x})$ become constants in this case as well. This implies that the
partial derivatives, the marginal energy cost related to usage of component
$x$, can be simplified to that constant. Which gives us back our third
assumption:

$\partial E_{x}/\partial U_{x}=f_{x}(U_{x})$ (14)

###  2.5 Potential Improvements and Research Questions

####  2.5.1 Empirical Estimations Of CPU Energy Usage

Note that these estimations do away with a logical formulation of
$f_{cpu}(U_{cpu})$, and instead empirically measure the energy consumption
when running on a certain CPU load. This data, collected through
experimentation, can then serve as input for statistical models to estimate
$f_{cpu}(U_{cpu}$).

####  2.5.2 Empirical Estimations Of Other Component Usage

Whereas there exist thus several initiatives that attempt to make sense of the
CPU energy consumption, I am not aware of any parties that do so for the other
components. Perhaps it is also not as relevant as CPU?

####  2.5.3 Empirical Validation of Assumptions

Although the assumptions laid out in the SDIA methodology are limiting, they
still provide means to obtain a number for energy consumption by a server.
Furthermore, it is not clear how inaccurate these numbers are, and if
scenarios exist where these numbers are valid nonetheless. These research
questions tie directly into the assumptions:

  * •

How realistic are the numbers for the allocation factors in the example?

  * •

Does linearity make sense? If so, in which cases? Which settings, which CPUs,
which architectures, or maybe even which kind of applications?

  * •

Can TDP actually be used as a proxy for CPU energy consumption at full
performance?

####  2.5.4 Empirical Validation of Idle Consumption

The last assumption has not been given enough attention in the formulas, but
it might be a swift assumption that systems without energy consumption by CPU,
memory, IO and networking can still be using energy. The system of course has
to maintain an operating modus, but it should be logical that this energy
consumption is negligible compared to the energy consumed by CPUs when they
are in usage.

Nonetheless, a measure of idle consumption might be extremely relevant in the
decision to lower load. If consumption when idling is relatively high,
powering off complete machines, or nodes would be a preferred strategy to
lower energy and emissions, rather than simply distributing and lowering the
load on each machine within a cluster.

  * •

When none of the components are consuming energy, what is the idling
consumption of a server?

  * •

Should idling consumption be incorporated into the formulas? If so, how?

##  3 Overhead: PUE

###  3.1 PUE, and Other Available Metrics

From an energy perspective, running something on a server is mostly a
conversion from electrical energy to thermal energy. Commonly, fans are used
to divert the heat away in most devices, but more sophisticated
infrastructure, like modern data centres., might use more advanced solutions,
like liquid cooling. No matter which form of equipment is being used, these
necessary appliances and components require energy to run as well, and
therefore can be regarded as overhead from a software perspective.

For data centres., the energy used by these overhead physical components is
most often reported as the Power Usage Effectiveness (PUE), which compares the
overall electrical energy used by a datacenter to the electrical energy that
actually is consumed by the hardware. Note thus that the measure is commonly
used when assessing overhead within data centres., but the same idea could be
applied on more individual devices.

Over time, the PUE metric has become the de-facto standard, and has also gone
some way to creating competition, driving efficiencies up as advertised PUE
values become lower. Standards for measurement and scope-determination have
been set up and most companies, especially hyperscalers boasting efficient PUE
numbers, provide somewhat detailed reports on the topic. For more information,
see the following whitepaper from the Green Grid Foundation [64].

Mathematically, you would get:

$E_{tot-cpu}=E_{cpu}*PUE$ (15)

Nonetheless, the metric has since then been under heavy scrutiny. As argued by
[7], PUE has some known issues:

  * •

PUE does not include the energy efficiency of the IT equipment

  * •

PUE does not reflect meteorological conditions of the facility. In colder
climates less energy might be needed for cooling purposes. Note that this
makes a crude comparison based only on PUE without climate reporting somewhat
meaningless.

  * •

The PUE metric is more useful for data centres that are operating at full IT
capacity, as this should mean that the supporting infrastructure is also
operating at its designed capacity

  * •

PUE does not include waste heat recycling

This is also why industry experts and researchers have been advocating for the
usage of additional metrics more focused on capturing a broader sense of
energy performance (see [54], [65]), while also pointing out that efforts to
improve the energy efficiency of the physical infrastructure of data centres
only lead to marginal improvements, and that focus should be on IT instead
[5]. These metrics include:

  * •

CUE: Carbon Usage Effectiveness

  * •

WUE: Water Usage Effectiveness

  * •

ERF: Energy Re-Usage Factor

  * •

ITEE: IT Equipment Efficiency

  * •

OEM: On-site Energy Matching (which focuses on on-site renewables)

Although data centres have become much more efficient over the past decades in
relation to all of these metrics ([42], [40]), and software is expected to
quickly become more carbon aware in the future, there remains an apparent
division between the two disciplines as shown by these metrics. In other
words, these metrics mostly serve to properly communicate efforts of data
centres to become more sustainable over longer periods of time, instead of
informing software developers of the instantaneous waste by the underlying
infrastructure when running their code.

Partially, this can be explained by the creation of abstraction layers between
the software and the hardware. But there are also commercial reasons behind
limiting the transparency around one’s operational efficiency. No matter the
reason, in the end we are all in the same boat and there are already companies
that intend to capitalise on this divide by offering a combination of
hardware, software, reporting and transparency. Next to that, the data centre
design space is rapidly innovating in several directions. Heat reusage, better
cooling and hardware custom designed for specific use-cases are expected to
become more and more prevalent in the future.

###  3.2 Potential Improvements and Research Questions

The biggest challenge lies in getting numbers out of data centres into the
heads of the consumers of those data centres. E.g. developers, engineers,
managers and the companies they work for. The solution not only lies in more
transparency, but might also require changes in data centre design. Not all
data centres allow for an effective measurement of the energy consumption of
individual software, and hence might be forced to turn to aggregates.

Next to that, although PUE aids in obtaining better estimates on energy
consumption of software, since it is generally aggregated over a year it makes
it impossible to get accurate instantaneous numbers.

Finally, the mathematics in this section have focused on PUE only, yet, with
new waste heat recovery methods finding their ways in modern data centres., it
might be better to account for this. Perhaps in a single metric representing
the amount of overhead or unrecoverable resources after running a certain
workload in some datacenter.

##  4 Momentaneous Energy Mix

The momentaneous Energy Mix represents the combination of energy sources
currently used to satisfy our society in its hunger for energy. In contrast to
PUE, we do have somewhat trustworthy numbers on the energy production side. We
can fairly trustworthy measure the amount of Energy generated by renewable
sources vs burning fossil fuels, and in turn the GHG emissions of the current
Energy Mix. Although there are some sources available (like ElectricityMaps
[15] and Wattime [67]), and even tools to easily retrieve data from these
sources ([24], [56], [18]), it is more challenging to properly connect to the
consumption of this energy. There is a discrepancy in time granularity between
energy production and energy consumption. That is, if it is only possible to
get reliable numbers on the emission of energy production per hour, you would
also need to aggregate the software energy consumption per hour in order to
perform calculations using the two variables. There is thus somewhat of a gap
between theory and practice here, as we would like to directly compare the
electricity Energy Mix of a particular moment with the consumption of that
energy in that particular moment.

In order to again provide a generic model and then introduce physical and
practical limitations we first integrate over time to later simplify to a
Riemann sum over the smallest granularity of the two measurements.
Mathematically, we would ideally get:

$kgCO2=PUE*\int_{0}^{T}GM(t)*E_{tot}(t)dt$ (16)

Wheres:

  * •

$T$ represents the total amount of time $GM(t)$ the Energy Mix at time $t$

  * •

$E_{tot-cpu}(t)$ the total energy consumption by the server

But, given that we can only get 30-minute Energy Mix updates:

$kgCO2=PUE*\sum_{t_{i}=1}^{T}GM(t_{i})\int_{=ti-1}^{t_{i}}E_{tot}(t_{j})$ (17)

The same can be said for the fact that we do not continuously measure energy
consumption, Hence:

$kgCO2=PUE*\sum_{t_{i}=1}^{T}GM(t_{i})\sum_{\tau=ti-1}^{t_{i}}E_{tot}(\tau)$
(18)

Notice thus that we might lose a bit of accuracy by smoothing out over time.
Also note that we focus here on CO2 emissions.

Given that we know the GHG emissions related to generating energy from both
renewables and non-renewables, we can get numbers for these other GHGs by
symmetry as well.

Finally, note that we generally focus on Energy (Joules) directly, but these
formulas can also be extended to either cover watts or Kilowatt-hours.

###  4.1 Potential Improvements and Research Questions

The discrepancy in time-granularity remains an issue, which we generally only
can solve by creating aggregations over time. Though, the same remark can be
made with regard to measuring Software Energy Consumption through RAPL, since
these tend to consist of more instantaneous snapshots rather than continuous
measurements.

Hence:

  * •

How reliable are Software Energy Measurement aggregations over time given that
their direct measurements can miss momentaneous peaks? Should we do better?
And can we do better?

  * •

How reliable is aggregating PUE over time? Should we do better?

  * •

How reliable is aggregating the Energy Mix over time? Should we do better? And
can we do better?

###  4.2 Average vs Marginal

Furthermore, there is ongoing debate between the usage of marginal CO2
emissions versus the average CO2 emissions when calculating the resulting CO2
emissions from having consumed some amount of energy. Note that this is a
completely different scientific field of its own, with great importance. After
all, modelling the modern energy grid with all its producers and consumers is
a complicated task. However, it is extremely important to accurately assess
the impact of globally changing consumption patterns due to, for example,
demand shifting and government policies.

Marginal emissions reflect the additional emissions coming from generating the
additional required energy on top of some baseline energy requirements. In
contrast, average emissions reflect the average emissions of all energy
producers delivering energy to the current Energy Mix.

In theory, as argued by [55], reducing the energy consumption would be to
reduce the energy production at those facilities that are operating at the
margin, or those facilities that are operating at the highest weighted
combination of fuel costs, emissions and operational costs. Therefore, it
would make sense to relate marginal energy demand to marginal energy supply.
In academic literature, the result of a change in systemic energy demand has
been shown to be more accurately measurable when using marginal emissions.
Since it is rather difficult to simulate the entire grid with all its
producers and consumers, these experiments are solely based on rather tightly
controlled simulations, like [16].

Despite the historic backing for marginal emissions over average emissions by
many academics, recently, (academic) criticism has increased. As argued by
[29] and [10], among others, stipulating which appliances and activities are
part of the marginal energy demand, and thereby establishing a order of merit
for energy consumption, is near impossible. In other words, should the
emissions of the fossil-fuel based power plant be attributed to one extra EV
charging cycle, your fridge or your neighbors jacuzzi? Stipulating the energy
demand of one of these appliances as marginal, would allocate all marginal
emissions, which often are fossil-fuel-based, to that appliance thereby
marking the energy consumption of the appliance as unnecessary or even
polluting.

In the inverse scenario, one where all energy demand is met by renewable
energy and marginal demand is to be met by burning fossil fuels, it is more
tempting to again conclude that marginal demand is met by marginal supply.
Yet, the same argument holds; it is impossible to establish an order of merit,
whether demand is new or existing.

Perhaps it is best to be pragmatic in this discussion. The final goal is to
reduce GHG emissions, and that might imply looking at averages in most
scenarios.

###  4.3 Power Purchasing Agreements and Guarantees of Origin

However, for the sake of transparency it might in some scenarios be best to
make use of averages, especially when dealing with the impact of Power
Purchasing Agreements between a large consumer and producer that come with
Guarantees of Origin (GO). In essence, these contracts form an agreement of
future delivery of energy against a certain price, whereas that energy is up
to some extent coming from renewable sources. Note that this does not directly
mean that the generated electrons that actually arrive at the consumer, have
been generated by those renewable sources. We are simply unable to track
electrons over the vast internationally interconnected grid, despite what some
companies might want you to believe [19]. In other words, without direct
connections to a renewable energy source, it is only possible to claim you are
running on 100% renewables if the actual renewable energy source is located
on-premise.

Nonetheless, for energy producers it is also interesting to offer these
agreements in order to invest in the production of these renewable energy
sources, even though the GOs are by some academics regarded as ineffective
policies [27]. This increase in the production of energy by renewable sources
can lower the average emissions over time of a region as a whole, rather than
just merely the purchaser of those GOs. If and only if some of the non-
renewable production can be replaced.

##  5 Embodied Carbon

The scientific field of calculating Embodied Carbon can be regarded as a field
on its own, more rooted into Life Cycle Assessments (LCA) than runtime
(operational) measurements. In short, Embodied Carbon involves attributing the
emitted carbon (or any other GHG emissions for that matter) related to
creating/manufacturing, transport, maintenance and disposal of the resource,
to the user of that resource. In a sense, you thus make the consumer of that
resource responsible for the emissions that were necessary to make and keep
that resource consumable in the first place, including proper disposal
afterwards.

These resources generally can take up a broad definition, but within the
domain of IT and data centres, we often talk about the hardware (server
racks), cabling and cooling networks, as well as the overhead physical
housing.

This also explains why measuring the Embodied Carbon asks for a completely
different approach. Fortunately, the theory is straightforward. We only need
to get these manufacturing-, transport- and recycling/disposal-related
emissions for each individual physical piece of equipment involved, and
attribute the amount of time it was used by a certain process proportional to
the total active lifetime of this individual physical piece of equipment.
Mathematically, you would thus sum over the total number of objects (n) and
their involved Embodied Carbon:

$kgCO2=\sum_{i=1}^{n}\left[(M_{o_{i}}+R_{o_{i}}+EoL_{o_{i}})(t_{c,o_{i}}/T_{o_{i}})\right]$
(19)

Whereas:

  * •

$n$: the total number of objects

  * •

$o_{i}$: a specific object, indexed as object $i$

  * •

$M_{o_{i}}$: The carbon emission involved during manufacturing of object $i$

  * •

$R_{o_{i}}$: The carbon emission involved during repairments/maintenance of
object $i$

  * •

$EoL_{o_{i}}$: The carbon emission involved during End-of-Life disposal of
object $i$

  * •

$t_{c,o_{i}}$: The total time some consumer $c$ consumed object $i$

  * •

$T_{o_{i}}$: The total time object $i$ remains in a consumable state (usage
phase lifespan)

In theory, these calculations are thus relatively straightforward. In
practice, a number of difficulties are likely to arise:

  * •

There might be thousands if not millions of objects involved, which all have
their own life-cycle related statistics.

  * •

These objects have different, potentially overlapping, usage phase lifespans
$T_{o_{i}}$.

  * •

Many resources can be shared by multiple consumers at the same time. Servers
are shared by multiple processes, racks by multiple servers, buildings and
cables by multiple racks, etc.

  * •

Transparency and auditability of the embedded carbon of these objects. It can
sometimes be difficult to get these numbers, if trustworthy at all.

Next to these difficulties, there are also a couple of problems likely to
arise, which might require a more holistic, perhaps even legislative,
solution:

  * •

Scope definitions should be extremely clear for these calculations to give
meaningful results. For many of the life cycle stages, these definitions are
not very strict and thus lead to reporters taking advantage of the
consequential leeway.

  * •

Who is responsible for the embodied carbon when an object is in the Usage-
phase, but actually not being used? E.g. when there actually is no consumer to
attribute to.

###  5.1 Life Cycle Statistics

For many products, especially those used in construction and physical
engineering, we already require manufacturers to disclose the carbon emitted
during manufacturing as well as the expected emissions during recycling or
waste disposal.

Although manufacturers might have an incentive to underestimate these carbon
emissions, especially when they are getting penalised for their emissions,
these should in general be relatively trustworthy. In the end, it is a
somewhat replicable physical process, so in theory it should well be possible
to audit these numbers.

Additionally, for objects with long life cycles it might be difficult to
predict how much carbon will be emitted during recycling or disposal. Who
knows where our recycling technology will be in 10 years, let alone 50 years
or more?

###  5.2 Different Lifespans

Depending on how far you are willing to go to get more accurate numbers, you
might need to account for replacing objects or parts that have shorter
lifespans, including back-up systems when these objects are part of a
continuous service without downtime. However, when you would be the consumer
of some object oi and its replacement part $o_{j}$, you do not have to
consider how these objects were related in the first place. You merely care
how long you have been a consumer of these objects.

###  5.3 Sharing of Objects and Resources

Properly attributing embodied carbon to individual consumers can already be
quite cumbersome when only dealing with the time- and object- dimensions.
However, ideally you also would want to properly attribute the sharing of
resources. If two consumers can make use of the same resource at the same
time, without a decrease in quality of service generated by that resource,
that resource would be used much more efficiently. So, evidently, this happens
a lot within data centres and it is for good reason data centre operators are
eager to see technologies like hypervisors and virtualization being developed
further to further push the efficiency (and in turn profitability) due to
sharing of resources. Hence, sharing of resources is quite common and it is
likely it will only be more common in the future.

However, proper carbon accounting and attributing embodied carbon downwards
becomes more difficult, especially as the sharing model becomes more complex
as well. In some moments in time, you might have an $x$ number of shared
consumers, and for other moments it might be different. Hence, if you intend
to properly account for the sharing of resources over time, you would need to
calculate this for each unit of time available.

Thus, for example, in order to accurately attribute the embodied carbon of a
building to one of the consumers that consumes one of the servers housed
within that building for an hour, you would need to know: The lifespan of the
building as a whole And how much of the total capacity is being used by that
consumer for each unit of time. Be it per second, or per hour, depending on
how much this would shift over time.

Mathematically, for each consumer you would get:

$kgCO2=\\\
\sum_{i=1}^{n}\left[(M_{o_{i}}+R_{o_{i}}+EoL_{o_{i}})(1/T_{o_{i}})\sum_{\tau=1}^{t_{c,o_{i}}}f_{c}({\tau,o_{i}})\right]$
(20)

Whereas:

  * •

$\tau$: denotes the individual steps in time between 1 and $t_{c,o_{i}}$, the
total time user u consumed resource $o_{i}$

  * •

$f_{c}({\tau},{o_{i}})$: denotes how much of resource oi is being consumed by
consumer $c$ at time . Note that this is a number between 0 and 1, a $1$
indicating that there is no sharing of resources whereas, for example, a $0.5$
would indicate that 50% of the resource is being consumed by consumer $c$, and
the other 50% by other consumers.

Hence, accounting for dynamic sharing of resources over time can greatly
complicate the calculations, and in turn introduce a lot of data requirements.
Both from a practical and theoretical point of view, there are objections to
the necessity of going this far. It might for example not even be feasible
from a practical point of view to know how many consumers of a certain
resource exist for each moment in time, or having to keep track of it might
introduce too much overhead. However, if you are working with reservations,
you would be able to better predict the sharing factors beforehand and you
would thus not need to track this actively over time.

From a theoretical point of view it is also questionable whether it would be
necessary in the first place. If the total number of consumers of a resource
is stable over time, as well as their sharing factors, you might be able to
get reasonably accurate numbers by the factors out over time.

###  5.4 Scope Definitions and Idle Time Usage

Whereas the difficulties are expected to be surmountable when reporting on
Embodied emissions, without proper legislative guidelines it might be much
more difficult to overcome the problem of scope and allocating Idle Time
within the usage phase.

Whereas scope definitions have been debated intensively over the last decade,
discussions around the attribution of embodied carbon during idle time usage
are just starting. And it is destined to be an interesting field filled with
controversy as well, with a lot of different use cases both within the IT and
data centres as well as the bigger public domain.

For example, take the football World Cup of 2022 hosted by Qatar organised by
FIFA. Qatar has built several new stadiums to host the matches, which are
expected to have a lifespan of only 60 years. Although there are some plans
here and there to reuse these stadiums for other events in the future, it is
unlikely these stadiums will be filled most of the time. FIFA, in their Carbon
Accounting report, has published the embodied carbon related to the stadiums
and the event as a whole. In the report, they only take responsibility for the
26 days during which the World Cup unfolded. The other days within the 60 year
lifespan are not taken into account. From a carbon accounting perspective, as
well as the formulas in the previous sections, they are right. However, it
remains suspicious given that these stadiums were built specifically for the
event. In this particular, who should be held responsible for those days when
the stadiums are not being used within their lifespan? FIFA? Qatar? Football
supporters visiting the event?

Of course, there must also be venues in the US and in Europe for which the
same can be said. Climate change is a global problem. Proper accounting of
embodied carbon as well as taking up responsibility for it, have global impact
as well.

###  5.5 Potential Improvements and Research Questions

  * •

As mentioned before, most of the work when calculating embodied carbon
emissions involves executing or obtaining proper Life Cycle Assessments (LCA).
In short, how reliable are the constants in the equations ($M_{o_{i}}$,
$R_{o_{i}}$, $EoL_{o_{i}}$, $T_{o_{i}}$).

  * •

If these are reliable, it is merely a matter of averaging out where possible.
The question then becomes for which scenarios this would be harmful and when
you are better off doing so. When can we average things out over time? How
dynamic are these resource consumption factors over time?

  * •

Next to that, which objects should be within scope? How can we further define
this properly such that everybody can start reporting adequately and
consistently?

  * •

Finally, who should be held responsible for idle time usage embodied carbon
emissions?

###  5.6 Note

Although this section has mostly focused on embedded carbon emissions, by
symmetry you could use the same equations for embedded emissions related to
other GHGs.

##  6 Conclusion

The field of sustainable IT has heavily been progressing during the time of
research and writing for this article, and it is hoped that this trend will
only accelerate in the future. Even though there are many different
stakeholders with different incentives, we are optimistic about the future,
especially because people show an overwhelming willingness to incorporate
sustainability in their work. We just need to be shown how sometimes.

## References

  * [1] A. Tarara, .  Spec Power Model.  https://github.com/green-coding-berlin/spec-power-model.
  * Andrae [2019] Andrae, A.S., 2019.  Projecting the chiaroscuro of the electricity use of communication and computing from 2018 to 2030.  Preprint , 1–23.
  * Andrae [2020] Andrae, A.S., 2020.  New perspectives on internet electricity use in 2030\.  Engineering and Applied Science Letters 3, 19–31.
  * Andrae and Edler [2015] Andrae, A.S., Edler, T., 2015\.  On global electricity usage of communication technology: trends to 2030.  Challenges 6, 117–157.
  * Bashroush and Lawrence [2020] Bashroush, R., Lawrence, A., 2020\.  Beyond pue: tackling it’s wasted terawatts.  Uptime Institute Available from: https://uptimeinstitute. com/beyond-pue-tackling-it’s-wasted-terawatts [Accessed 11 June 2020] .
  * Belkhir and Elmeligi [2018] Belkhir, L., Elmeligi, A., 2018\.  Assessing ict global emissions footprint: Trends to 2040 & recommendations.  Journal of cleaner production 177, 448–463.
  * Brady et al. [2013] Brady, G.A., Kapur, N., Summers, J.L., Thompson, H.M., 2013\.  A case study and critical assessment in calculating power usage effectiveness for a data centre.  Energy Conversion and Management 76, 155–161.
  * Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change [2022] Contribution of Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, 2022\.  Climate change 2022: Impacts, adaptation, and vulnerability.  .
  * Cooper et al. [2010] Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R., 2010.  Benchmarking cloud serving systems with ycsb, in: Proceedings of the 1st ACM symposium on Cloud computing, pp. 143–154.
  * Corradi [2022] Corradi, O., 2022.  Marginal vs average: which one to use for real-time decisions?  URL: https://www.electricitymaps.com/blog/marginal-vs-average-real-time-decision-making.
  * Cozzi et al. [2017] Cozzi, L., Turk, D., Abergel, T., Bartos, J., Bellevrat, E., Bennett, S., Berly, T., Bouckaert, S., Dulac, J., Alvarez, C.F., et al., 2017\.  Digitalization and Energy.  OECD.
  * [12] Davy, B., .  Estimating AWS EC2 Instances Power Consumption.  https://medium.com/teads-engineering/estimating-aws-ec2-instances-power-consumption-c9745e347959.
  * Davy [2021] Davy, B., 2021.  https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac.
  * Desrochers et al. [2016] Desrochers, S., Paradis, C., Weaver, V.M., 2016.  A validation of dram rapl power measurements, in: Proceedings of the Second International Symposium on Memory Systems, Association for Computing Machinery, New York, NY, USA. p. 455–470.  URL: https://doi.org/10.1145/2989081.2989088, doi:10.1145/2989081.2989088.
  * Electricitymaps [2023] Electricitymaps, 2023.  Electricitymaps.  URL: https://app.electricitymaps.com/.
  * Elenes et al. [2022] Elenes, A.G.N., Williams, E., Hittinger, E., Goteti, N.S., 2022\.  How well do emission factors approximate emission changes from electricity system models?  Environmental Science & Technology 56, 14701–14712.  URL: https://doi.org/10.1021/acs.est.2c02344, doi:10.1021/acs.est.2c02344. pMID: 36153999.
  * Ferreboeuf and Lean [2019] Ferreboeuf, H., Lean, I., 2019\.  Towards digital sobriety.  The Shift Project. The Carbon Transition Think Tank .
  * Foundation [2023] Foundation, G.W., 2023.  Grid intensity go.  URL: https://github.com/thegreenwebfoundation/grid-intensity-go.
  * Frost [2023] Frost, R., 2023.  Why have irish energy companies been told to drop ’misleading’ 100% renewable claims?  URL: https://www.euronews.com/green/2023/02/10/why-have-irish-energy-companies-been-told-to-drop-misleading-100-renewable-claims?utm_source=climateActionTech&utm_medium=email.
  * García-Martín et al. [2019] García-Martín, E., Lavesson, N., Grahn, H., Casalicchio, E., Boeva, V., 2019.  How to measure energy consumption in machine learning algorithms, in: ECML PKDD 2018 Workshops: Nemesis 2018, UrbReas 2018, SoGood 2018, IWAISe 2018, and Green Data Mining 2018, Dublin, Ireland, September 10-14, 2018, Proceedings 18, Springer. pp. 243–255.
  * Geist [2020] Geist, J., 2020.  Did the shift project really overestimate the carbon footprint of online video? our analysis of the iea and carbonbrief articles.  The Shift Project website .
  * Goel et al. [2012] Goel, B., McKee, S.A., Själander, M., 2012.  Techniques to measure, model, and manage power, in: Advances in Computers. Elsevier. volume 87, pp. 7–54.
  * [23] Green Software Foundation, .  Software carbon intensity specification.  URL: \url{https://github.com/Green-Software-Foundation/sci/blob/main/Software_Carbon_Intensity/Software_Carbon_Intensity_Specification.md}.
  * Green Software Foundation [2023] Green Software Foundation, 2023.  Carbon-aware sdk.  URL: \url{https://github.com/Green-Software-Foundation/carbon-aware-sdk}.
  * Hackenberg et al. [2015] Hackenberg, D., Schöne, R., Ilsche, T., Molka, D., Schuchart, J., Geyer, R., 2015\.  An energy efficiency feature survey of the intel haswell processor, in: 2015 IEEE International Parallel and Distributed Processing Symposium Workshop, pp. 896–904.  doi:10.1109/IPDPSW.2015.70.
  * Hähnel et al. [2012] Hähnel, M., Döbel, B., Völp, M., Härtig, H., 2012\.  Measuring energy consumption for short code paths using rapl.  SIGMETRICS Perform. Eval. Rev. 40, 13–17.  URL: https://doi.org/10.1145/2425248.2425252, doi:10.1145/2425248.2425252.
  * Ákos Hamburger [2019] Ákos Hamburger, 2019.  Is guarantee of origin really an effective energy policy tool in europe? a critical approach.  Society and Economy Soc Ec 41, 487 – 507.  URL: https://akjournals.com/view/journals/204/41/4/article-p487.xml, doi:https://doi.org/10.1556/204.2019.41.4.6.
  * Henning [2006] Henning, J.L., 2006.  Spec cpu2006 benchmark descriptions.  SIGARCH Comput. Archit. News 34, 1–17.  URL: https://doi.org/10.1145/1186736.1186737, doi:10.1145/1186736.1186737.
  * Hoekstra [2023] Hoekstra, A., 2023.  Marginal emissions are a hype in academia.  URL: https://twitter.com/AukeHoekstra/status/1620525739803291648.
  * Hubblo.io [2023] Hubblo.io, 2023.  Scaphandre Documentation.  URL: https://hubblo-org.github.io/scaphandre-documentation/.
  * Intel [2022] Intel, 2022.  Running average power limit energy reporting / cve-2020-8694 , cve-2020-8695 / intel-sa-00389.  URL: https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/advisory-guidance/running-average-power-limit-energy-reporting.html.
  * Intel [2023] Intel, 2023.  Intel Power Gadget.  URL: https://www.intel.com/content/www/us/en/developer/articles/tool/power-gadget.html.
  * Issa [2016] Issa, J., 2016.  Performance characterization and analysis for hadoop k-means iteration.  Journal of Cloud Computing 5, 1–15.
  * Kaivalya M. Dixit [1993] Kaivalya M. Dixit, 1993.  Overview of the SPEC Benchmarks, in: The Benchmark Handbook for Database and Transaction Systems (2nd Edition). Morgan Kaufmann. chapter 9, p. .
  * Kamiya [2022] Kamiya, G., 2022.  Factcheck: What is the carbon footprint of streaming video on Netflix?  https://www.carbonbrief.org/factcheck-what-is-the-carbon-footprint-of-streaming-video-on-netflix/.
  * Kavanagh et al. [2016] Kavanagh, R., Armstrong, D., Djemame, K., 2016.  Accuracy of energy model calibration with ipmi, in: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pp. 648–655.  doi:10.1109/CLOUD.2016.0091.
  * Kavanagh and Djemame [2019] Kavanagh, R., Djemame, K., 2019\.  Rapid and accurate energy models through calibration with ipmi and rapl.  Concurrency and Computation: Practice and Experience 31, e5124.
  * Khan et al. [2018] Khan, K.N., Hirki, M., Niemi, T., Nurminen, J.K., Ou, Z., 2018.  Rapl in action: Experiences in using rapl for power measurements.  ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS) 3, 1–26.
  * King [2023] King, C., 2023.  Powerstat Bionic Manpage.  URL: https://manpages.ubuntu.com/manpages/bionic/man8/powerstat.8.html.
  * Koomey et al. [2010] Koomey, J., Berard, S., Sanchez, M., Wong, H., 2010\.  Implications of historical trends in the electrical efficiency of computing.  IEEE Annals of the History of Computing 33, 46–54.
  * Koomey and Masanet [2021] Koomey, J., Masanet, E., 2021\.  Does not compute: Avoiding pitfalls assessing the internet’s energy and carbon impacts.  Joule 5, 1625–1628.
  * Koomey et al. [2009] Koomey, J.G., Berard, S., Sanchez, M., Wong, H., 2009\.  Assessing trends in the electrical efficiency of computation over time.  IEEE Annals of the History of Computing 17\.
  * Malmodin and Lundén [2018] Malmodin, J., Lundén, D., 2018\.  The energy and carbon footprint of the global ict and e&m sectors 2010–2015.  Sustainability 10, 3027\.
  * Masanet et al. [2020] Masanet, E., Shehabi, A., Lei, N., Smith, S., Koomey, J., 2020.  Recalibrating global data center energy-use estimates.  Science 367, 984–986.
  * MLCO2 [2023] MLCO2, 2023.  CodeCarbon Documentation.  URL: https://github.com/mlco2/codecarbon.
  * Morley et al. [2018] Morley, J., Widdicks, K., Hazas, M., 2018.  Digitalisation, energy and data demand: The impact of internet traffic on overall and peak electricity consumption.  Energy Research & Social Science 38, 128–137.
  * Mytton and Ashtine [2022] Mytton, D., Ashtine, M., 2022\.  Sources of data center energy estimates: A comprehensive review.  Joule .
  * Noureddine [2023] Noureddine, A., 2023.  JoularJX Documentation.  URL: https://github.com/joular/joularjx.
  * Pandruvada [2014] Pandruvada, S., 2014.  Running average power limit.  URL: https://01.org/blogs/2014/running-average-power-limit-\%E2\%80\%93-rapl.
  * Paniego et al. [2018] Paniego, J.M., Gallo, S., Pi Puig, M., Chichizola, F., De Giusti, L., Balladini, J., 2018\.  Analysis of rapl energy prediction accuracy in a matrix multiplication application on shared memory, in: Computer Science–CACIC 2017: 23rd Argentine Congress, La Plata, Argentina, October 9-13, 2017, Revised Selected Papers 23, Springer. pp. 37–46.
  * Pasek et al. [2023a] Pasek, A., Vaughan, H., Starosielski, N., 2023a.  The world wide web of carbon: Toward a relational footprinting of information and communications technology’s climate impacts.  Big Data & Society 10, 20539517231158994\.  URL: https://doi.org/10.1177/20539517231158994, doi:10.1177/20539517231158994.
  * Pasek et al. [2023b] Pasek, A., Vaughan, H., Starosielski, N., 2023b.  The world wide web of carbon: Toward a relational footprinting of information and communications technology’s climate impacts.  Big Data & Society 10, 20539517231158994\.
  * Rteil et al. [2022] Rteil, N., Bashroush, R., Kenny, R., Wynne, A., 2022\.  Interact: It infrastructure energy and cost analyzer tool for data centers.  Sustainable Computing: Informatics and Systems 33, 100618.
  * Salim [2020] Salim, M., 2020.  How green standards are changing data centre design and operations.  Synthesis .
  * Schram et al. [2019] Schram, W., Lampropoulos, I., AlSkaif, T., Van Sark, W., Helfert, M., Klein, C., Donnellan, B., et al., 2019.  On the use of average versus marginal emission factors, in: SMARTGREENS 2019-Proceedings of the 8th International Conference on Smart Cities and Green ICT Systems, SciTePress. pp. 187–193.
  * Source [2023] Source, M.O., 2023.  Kubernetes carbon intensity exporter.  URL: https://github.com/Azure/kubernetes-carbon-intensity-exporter/.
  * SPEC [2014] SPEC, 2014.  Power and Performance Benchmark Technology V2.2.  PowerandPerformanceBenchmarkMethodologyV2.2.
  * SPEC [2018] SPEC, 2018.  SPEC Cloud IaaS 2018.  https://www.spec.org/cloud_iaas2018/.
  * Sun et al. [2019] Sun, Y., Agostini, N.B., Dong, S., Kaeli, D., 2019\.  Summarizing cpu and gpu design trends with product data.  arXiv preprint arXiv:1911.11313 .
  * Sustainable Computing [2023] Sustainable Computing, 2023.  Kubernetes Efficient Power Level Exporter (Kepler).  URL: https://sustainable-computing.io/.
  * Tarara [2023] Tarara, A., 2023.  Green Metrics Tool.  URL: https://github.com/green-coding-berlin/green-metrics-tool.
  * Thoughtworks [2023] Thoughtworks, 2023.  Cloud Carbon Footprint.  URL: https://www.cloudcarbonfootprint.org/.
  * UPPA [2023] UPPA, G., 2023.  Welcome to AI Power Meter’s Documentation!  URL: https://greenai-uppa.github.io/AIPowerMeter/.
  * V. Avelar [2012] V. Avelar, D. Azevedo, A.F., 2012.  PUE: A Comprehensive Examination of the Metric.
  * Van De Voort et al. [2017] Van De Voort, T., Zavrel, V., GALDIZ, I.T., Hensen, J., 2017\.  Analysis of performance metrics for data center efficiency.  REHVA Journal .
  * van de Ven [2023] van de Ven, A., 2023.  Powertop.  URL: https://wiki.archlinux.org/title/powertop.
  * wattime [2023] wattime, 2023.  Wattime.  URL: https://www.watttime.org/.