# Supply chain emission estimation using large language models

Ayush Jain<EMAIL_ADDRESS>,  Manikandan Padmanaban<EMAIL_ADDRESS>IBM Research Labs India ,  Jagabondhu Hazra<EMAIL_ADDRESS>,  Shantanu
Godbole<EMAIL_ADDRESS>IBM Research Labs India560045 and  Kommy
Weldemariam<EMAIL_ADDRESS>IBM Research Labs USA

###### Abstract.

Large enterprises face a crucial imperative to achieve the Sustainable
Development Goals (SDGs), especially goal 13, which focuses on combating
climate change and its impacts. To mitigate the effects of climate change,
reducing enterprise Scope 3 (supply chain emissions) is vital, as it accounts
for more than 90% of total emission inventories. However, tracking Scope 3
emissions proves challenging, as data must be collected from thousands of
upstream and downstream suppliers. To address the above mentioned challenges,
we propose a first-of-a-kind framework that uses domain-adapted NLP foundation
models to estimate Scope 3 emissions, by utilizing financial transactions as a
proxy for purchased goods and services. We compared the performance of the
proposed framework with the state-of-art text classification models such as
TF-IDF, word2Vec, and Zero shot learning. Our results show that the domain-
adapted foundation model outperforms state-of-the-art text mining techniques
and performs as well as a subject matter expert (SME). The proposed framework
could accelerate the Scope 3 estimation at Enterprise scale and will help to
take appropriate climate actions to achieve SDG 13.

scope 3 emission, large language models, SDG goals 13

††conference: August 06-10, 2023; Long Beach, CA;

##  1\. Introduction

The United Nations (UN) Sustainable Development Goals (SDGs) provide a
framework for achieving a better and more sustainable future for all, with 17
goals and 169 targets adopted by all UN member states in 2015. Among these
goals, SDG 13 focuses on combating climate change and its impacts, with a
primary target of integrating climate change measures into national policies,
strategies, and planning (Nations, 2022). Unfortunately, the world remains off
track in meeting the Paris Agreement’s target of limiting the temperature rise
to 1.5∘C above pre-industrial levels and reaching net-zero emissions by 2050
(Nations, 2021), with a projected temperature rise of around 2.7∘C above pre-
industrial levels by 2100 (Tracker, 2022). To achieve these targets, it is
critical to engage non-state actors like enterprises, who have pledged to
reduce their GHG emissions, and have significant potential to drive more
ambitious actions towards climate targets than governments (Li et al., 2019).
However, a lack of high-quality data and insights about an enterprise’s
operational performance can create barriers to reducing emissions. AI assisted
tools can play a vital role in addressing these concerns and enabling an
enterprise’s decarbonization journey (Vinuesa et al., 2020).

The GHG Protocol Corporate Standard (Protocol et al., 2004) provides a
systematic framework for measuring an enterprise’s GHG emissions, classifying
them into three scopes. Scope 1 (S1) emissions are direct emissions from owned
or controlled sources, while Scope 2 (S2) emissions are indirect emissions
from the generation of purchased energy. Scope 3 (S3) emissions are all
indirect emissions that occur in the value chain of the reporting company,
including both upstream and downstream emissions. To provide companies with a
more detailed picture of their S3 emissions, they are further divided into 15
distinct reporting categories, such as Purchased Goods and Services, Capital
Goods, and Business Travel. However, since S3 emissions occur outside of an
enterprise’s direct control or ownership, tracking and measuring them can be
extremely challenging.

Despite the importance of S3 emissions in reducing GHG emissions, most
enterprises have focused on reducing their S1 and S2 emissions, leaving S3
emissions largely unabated. However, for many enterprises, S3 emissions make
up the majority of their emission inventories. A report by (Li et al., 2019)
found that reducing global S3 emissions by an additional 54% from current
baseline scenarios could help achieve a climate reduction target of limiting
temperature rise to 1.75∘ C above pre-industrial levels by 2100, instead of
the projected 2.7∘ C. Since S3 emissions often overlap with other enterprises’
emission inventories, reducing S3 emissions not only has the potential to
mitigate the impact of climate change but can also provide significant
business benefits (Farsan et al., 2018).

Foundation models are a recent breakthrough in the field of artificial
intelligence, comprising large models trained on vast datasets using self-
supervised learning techniques. Once trained, these models can be fine-tuned
for various downstream tasks, yielding superior performance compared to
traditional machine learning and deep learning models. While foundation models
have demonstrated remarkable success in numerous domains, their application in
the context of climate and sustainability has yet to be fully explored. The
potential of foundation models to aid in reducing greenhouse gas emissions and
achieving sustainable development goals calls for further investigation.

Prior research has started to explore the potential of foundation models in
the domain of climate and sustainability. For instance, (Nugent et al., 2021)
used a domain-specific language model pre-trained on business and financial
news data to identify Environmental, Social and Governance (ESG) topics.
(Luccioni et al., 2020) employed a custom transformer-based NLP model to
extract climate-relevant passages from financial documents. Similarly,
(Corringham et al., [n. d.]) investigated the application of pre-trained
transformers based on the BERT architecture to classify sentences in a dataset
of climate action plans submitted to the United Nations following the 2015
Paris Agreement. In addition, (Lacoste et al., 2021) proposed the development
of a benchmark comprising several downstream tasks related to climate change
to encourage the creation of foundation models for Earth monitoring. However,
none of these studies have focused on utilizing foundation models for
estimating S3 emissions.

In this paper, we proposed a first-of-a-kind framework for estimating Scope 3
emission leveraging large language models by classifying transaction/ledger
description of purchase good and services into US EPA EEIO commodity classes.
We conduct extensive experiments with multiple state of the art large language
models (roberta-base, bert-base-uncased and distilroberta-base-climate-f) as
well as non-foundation classical approaches (TF-IDF and Word2Vec) and
investigate the potential of foundation models for estimating S3 emissions
using financial transaction records as a proxy for goods and services. We
compare the performance of conventional models with foundation models for
commodity recognition and mapping of financial transaction records into United
States Environmentally-extended input-output (US EEIO) commodity classes. We
also present a case study based on a sample enterprise financial ledger
transaction data to demonstrate the proposed framework.

##  2\. Background

###  2.1. Scope 3 challenges

Scope 3 emissions along the value chain account for around 80% of a company’s
greenhouse gas impacts, which makes it important for companies to account for
their Scope 3 emissions along with their Scope 1 and 2 emissions in order to
develop a full GHG emissions inventory. This is also required to enable
companies to understand their full value chain emissions and to focus their
efforts on the greatest GHG reduction opportunities. Most of the largest
companies in the world now account and report the emissions from their direct
operations (Scopes 1 and 2). However, merely few companies are currently
reporting their Scope 3 emissions for limited categories. The primary reason
behind this contrast is the immense complexity associated with tracking and
accounting of Scope 3 emissions. Some of the primary challenges faced by
enterprises today are as follows:

  * •

Daunting data collection process: Organizations struggle to collect relevant
and sufficiently granular primary data and to manage the amount of data needed
to calculate Scope 3 emissions.

  * •

Lack of Cooperation: In order to successfully manage their Scope 3 emissions,
enterprises need a certain extent of collaboration with third parties like
upstream suppliers, lessors/lessees, employees or customers. Enterprises today
face a lack of cooperation along the value chain, which hinders them from
successfully managing Scope 3 emissions.

  * •

Lack of Transparency: The lack of transparency regarding the relevance of
Scope 3 emissions is another major challenge for many companies. They may also
lack transparency into their upstream and downstream partners’ data collection
and calculation processes, which reduces the overall quality of information
exchanged.

  * •

Large Set of Stakeholders: Scope 3 involves a large set of stakeholders across
processes, including the upstream and downstream value chain. This makes the
task of data collection more complex and also makes it challenging for
enterprises to track and measure their Scope 3 emissions.

  * •

Lack of Personnel Resources: The calculation of Scope 3 emissions requires
personnel, resources, expertise, and data management and quality processes.
Enterprises need a good management and leadership support in order to bring
alignment which in itself is not a simple task.

Figure 1. Framework for estimating Scope3 using large language model

###  2.2. Scope 3 calculation methodologies

The GHG Protocol Corporate Standard (Protocol et al., 2004) recommends the use
of Life Cycle Assessment (LCA) approach (Vigon et al., 2020) or
environmentally Extended Input-Output Analysis (EIOA) method (Miller and
Blair, 2009) for Scope 3 reporting and estimation. The LCA approach evaluates
the environmental footprints of a product or activity by considering the
entire life-cycle of the processes involved starting from raw material
extraction to the disposal. It requires the detailed knowledge about the
activities and services involved in the supply chain and its associated
emission factors. But the large companies have tens of thousand of products
and services, and it is very complex and impractical to collect the detailed
physical activity data across the supply chain. Alternatively, the most widely
adopted method is environmentally Extended Input-Output Analysis (EIOA)
method. It is based on the economic model called Input-Output Analysis (IOA)
proposed by (Leontief, 1970), which captures the inter and intra-trade
relations between various sectors of economy across different countries. This
economic model was extended to include the environmental impact called
environmentally Extended Input-Output model, which estimates the amount of
carbon footprints for delivering the one-dollar output by the sectors or
products. It only requires to collect the value-based (monetary values)
activity data of the company. It mainly consists of purchasing data and it is
broadly classified into few product group or economic sectors. The emission
factors for every dollar spend on each group of products or economic sectors
can be determined from the EEIO framework (Schmidt et al., 2022).

Apart from these activity-data based and spend-based approaches, few research
works have also focused on using machine learning techniques for estimating
Scope 3 emissions. (Serafeim and Velez Caicedo, 2022) built ML regressor
models to predict Scope 3 emissions using widely available financial statement
variables, Scope 1&2 emissions, and industrial classifications. (Nguyen et
al., 2022) explored the quality of Scope 3 emission data in terms of
divergence and composition, and the performance of machine-learning models in
predicting Scope 3 emissions, using emission datasets from different data
providers.

###  2.3. USEEIO dataset

US Environmentally-Extended Input-Output (USEEIO) is a life cycle assessment
(LCA) model that tracks the economic and environmental flows of goods and
services in the United States. It provides comprehensive dataset and
methodology that combines economic input-output analysis with environmental
information to quantify environmental impacts associated with economic
activities. USEEIO is a free and open-source dataset that is available for
download from the US Environmental Protection Agency website.

USEEIO classifies commodities into over 380 categories of categories of goods
and services that share similar environmental characteristics, which are
called commodity classes. USEEIO provides emission factors for for these
commodity classes, which are are to estimate environmental impact from spend
data. These commodity classes are designed to align with the North American
Industry Classification System (NAICS) and the Bureau of Economic Analysis
(BEA) codes, where BEA codes aggregate multiple NAICS codes into higher level
of industry descriptions. This alignment ensures consistency and facilitates
the integration of USEEIO with existing economic and environmental datasets.
Apart from the detailed commodity classes, USEEIO also provides emission
factors for 66 summary commodity classes.

###  2.4. Why Foundation Model

In last couple of years, big supply chain players are trying their best to
collect primary data from their suppliers through multiple mechanism e.g.
supplier selections, financial incentives, etc. Unfortunately, there is no
notable progress due to the challenges mentioned in Section 2.1. Therefore, it
is imperative to use proxy data such as financial transactions for Scope 3
calculation. Spend data is easily available in any organization and is a good
proxy of quantity of goods/services purchased and embedded emission.

While spend based Scope 3 emission estimation has an opportunity to address
this complex problem, there are some challenges as follows:

  * •

Commodity recognition: Purchased products and services are described in
natural languages in various forms. Commodity recognition from purchase
orders/ledger entry is extremely hard.

  * •

Commodity mapping: There are millions of products and services. Spend based
emission factor is not available for individual product or service categories.
Organizations like US Environmental Protection Agenecy (EPA) or Organization
for Economic Cooperation and Development (OECD) provide spend based emission
factors for limited product/service categories. Manual mapping of the
commodity/service to product/service category is extremely hard if not
impossible.

Recent advances in deep learning based foundation models for natural language
processing (NLP) have proven to outperform conventional machine learning
models across a broad range of NLP classification tasks when availability of
labeled data is insufficient or limited. Leveraging large pre-trained NLP
models with domain adaptation with limited data has an immense potential to
solve the Scope3 problem in climate and sustainability.

##  3\. Proposed method

We propose a first-of-a-kind AI based commodity name extraction and mapping
service for scope3 emission estimation leveraging large language models. The
framework is designed to process any Enterprise ledger transaction record
written in natural languages to EEIO product/service category for estimating
Scope 3 emissions. Figure 1 presents the proposed framework for estimating
scope 3 emission using large language model. The framework consists of 4
modules i.e. data preparation, domain adaptation, classification, and emission
computation. In the data preparation module, we thoughtfully created around
18000 examples of expenses (250-300 samples per class) for the 66 unique US
EPA EEIO broad summary commodity classes. The samples are generated as per the
International standard industrial classification of all economic activities
(ISIC) with adequate representation from each sub-classes. In the subsequent
module, we have chosen pre-trained foundation models (e.g. BERT, RoBERTa,
ClimateBERT) and fine-tuned the foundation models for 66 commodity classes
using the labeled samples with 70:20:10 train-validation-test split. Once the
model is adapted, we evaluate its performance with reasonable test samples. We
identified the classes with low performance and further fine-tuned the model
with additional training samples for those classes. Once the domain adaptation
is completed, the fine-tuned FM is deployed for inferencing on unknown data
set. Now the fine-tuned model can take any ledger description written in
natural language as input and provide the best matched industry/commodity
class. Once, the commodity mapping is completed, the frame work can compute
Scope 3 emission for each purchase line item using given EEIO emission
factors.

This framework will help to avoid the humongous manual efforts and will
accelerate the Scope 3 estimation at Enterprise scale. As financial
transactions are easily accessible, the proposed method could make the Scope 3
calculation way more affordable at enterprise scale.

Table 1. Values of hyperparameter varied during experimentation Hyperparameter | Values
---|---
max_length | 64, 128, 256, 512
Learning rate | 5e-5, 5e-6
Table 2. Results for Zero-shot classification. TS- Semantic Text Similarity, Title/Description - Whether title or description of commodity classes was used Approach | Model | Title/Description | F1 Score (test)
---|---|---|---
TS | all-mpnet-base-v2 | Title | 27.9%
Description | 43.7%
all-MiniLM-L6-v2 | Title | 27.5%
Description | 40.1%
all-MiniLM-L12-v2 | Title | 27.5%
Description | 40.9%

##  4\. Experiment

We experiment with different training strategies including our proposed
approach for the task of commodity classification in order to evaluate and
compare their performance on the given problem.

###  4.1. Using Zero-shot classification

We tried zero-shot learning to evaluate if we can perform classification of
transaction data to EEIO commodity classes, without using any domain specific
training data.

####  4.1.1. Using Semantic Text similarity

(Balaji et al., 2023) proposed carbon fooptprint of household products using
Zero-shot semantic text similarity. This technique aims to assign predefined
labels to text without any explicit training examples. It leverages pre-
trained language models and sentence transformers, which are deep learning
architectures specifically designed to encode textual information into fixed-
length vector representations. By employing these models, the semantic
relatedness between input text and commodity classes can be effectively
measured by means of a semantic similarity score between the embeddings of
input text and commodity classes.

We tried few experimental settings to test Semantic Text similarity approach
for our task-

  * •

Commodity class titles vs Commodity class descriptions: As mentioned above,
this approach works by finding semantic similarity between the embeddings of
input text and commodity classes. For computing the embedding of commodity
classes we tried both the titles of the commodity classes as well as their
descriptions of their NAICS codes provided in the US Bureau of Labor
Statistics Website 111https://www.bls.gov. For commodity classes which were
comprised of more than one NIACS code, we concatenated the descriptions of the
individual NAICS codes to create the commodity class description. For example,
a commodity class Food and beverage and tobacco products (Commodity code
311FT) in USEEIO is comprised of 2 NAICS classes -  Food Manufacturing and
Beverage and Tobacco Product Manufacturing, having NAICS codes 311 and 312
respectively.

  * •

Sentence transformer models: We additionally experimented with different open
source sentence-transformer models available in sentence-transformers python
library (Reimers and Gurevych, 2019) to compute embeddings for the input
transaction text and commodity names/descriptions. We tried all-mpnet-base-v2,
all-MiniLM-L12-v2 and all-MiniLM-L6-v2 models.

We used cosine similarity between the embeddings of input text and commodity
classes as the measure of semantic similarity, for all the experimental
settings mentioned above.

###  4.2. Supervised learning using classical model (SL)

We experimented with supervised learning approach using classical ML model. It
required feature vectorization of the input text in order to perform model
training. We tried TF-IDF (Term Frequency-Inverse Document Frequency) and
Word2Vec for feature vectorization of the input transaction texts.

####  4.2.1. TF-IDF

TF-IDF represents the importance of a term in a document, taking into account
both its frequency in the document (TF) and its rarity across the entire
corpus (IDF) and is a widely used technique in Information Retrieval and Text
Mining We first performed feature extraction and calculated the TF-IDF values
for each term in the training corpus. Then we perform feature vectorization
wherein each input transaction text is converted to a numerical feature vector
based on the TF-IDF values. We used the TF-IDF feature vector for training ML
classifier model.

####  4.2.2. Word2Vec

Word2Vec introduced in (Mikolov et al., 2013), is a popular algorithm for
generating word embeddings, which are dense vector representations of words in
a high-dimensional space. It learns to encode semantic relationships between
words by training on a large corpus of text. We used Word2Vec to create
feature representation of the input transaction data for model training. This
was done by computing the average of word embeddings for all the words in a
given transaction text.

We trained random forest classifier models for both TF-IDF and Word2Vec based
feature vectors representation.

###  4.3. Supervised fine-tuning for Scope3 (SFT3)

We additionally experimented with fine-tuning encoder based Large Language
Models (LLMs) for classifying transaction data into commodity classes. These
models are pre-trained using masked language modeling (MLM) and next sentence
prediction (NSP) objectives. We tried the following commonly used models
available on Huggingface.

##### bert-base-uncased

The BERT (Bidirectional Encoder Representations from Transformers) is a
popular pre-trained model developed by (Devlin et al., 2019). BERT-base-
uncased is uncased version of BERT model which is trained on a massive amount
of textual data from books, articles, and websites. The training process
involves a transformer-based architecture, which enables BERT to capture
bidirectional dependencies between words and context. BERT-base-uncased
consists of 12 transformer layers, each with a hidden size of 768, and has 110
million parameters.

##### roberta-base

RoBERTa (Robustly Optimized BERT pretraining Approach) is a variant of the
BERT (Bidirectional Encoder Representations from Transformers) model developed
by (Liu et al., 2019). Similar to BERT, the RoBERTa-base also utilises dynamic
masking during pre-training where tokens are randomly masked out. However,
RoBERTa-base extends this approach by training with larger batch sizes, more
iterations, and data from a more diverse range of sources. It also utilises a
larger vocabulary and removes the next sentence prediction objective used in
BERT, focusing solely on the masked language modeling objective. It excels in
understanding the context and semantics of text, and captures subtle
relationships and fine-grained nuances in language. It consists of 12
transformer layers, each with a hidden size of 768, and has 125 million
parameters.

##### climatebert/distilroberta-base-climate-f

The ClimateBERT language model is additionally pre-trained, on top of
DistilRoBERTa model, on text corpus comprising climate-related research paper
abstracts, corporate and general news and reports from companies (Webersinke
et al., 2022). It has been shown improve the performance of various climate
related downstream tasks (Garrido-Merchán et al., 2023). The model has 6
transformer layers, 768 dimension and 12 heads, totalizing 82M parameters
(compared to 125M parameters for RoBERTa-base).

We used transformer (Wolf et al., 2020) library from Huggingface which allows
to easily load and fine-tune pre-trained models from Huggingface. We
experimented with varying the max_length parameter which determines the
maximum length of the input sequences that the model can handle. We also
experimented with lowering the default learning rate that is provided in the
library. Table 1 shows the values of different values of hyperparameters that
were tried as part of the experimentations. We evaluated the results on the
model checkpoint with best validation loss by setting load_best_model_at_end
argument to be True.

Table 3. Results for Supervised learning using classical model (SL). TF-IDF: Model was trained on TF-IDF based feature vectors, Word2Vec: Model was trained on Word2Vec based feature vectors | F1 (test)
---|---
TF-IDF | 69%
Word2Vec | 72%
Table 4. Results for supervised fine-tuning ($\alpha$ = 5e-5) Model | max_length | F1 Score (test)
---|---|---
roberta-base | 64 | 86.17%
128 | 86.8%
256 | 87.2%
512 | 86.4%
bert-base-uncased | 64 | 86.6%
128 | 86.38%
256 | 86.32%
512 | 86.44%
distilroberta-base-climate-f | 64 | 85.48%
128 | 85.4%
256 | 84.5%
512 | 85.1 %
Table 5. Results for lower learning rate ($\alpha$ = 5e-6) Model | max_length | F1 (test)
---|---|---
roberta-base | 512 | 87.19%
bert-base-uncased | 512 | 87.14%
distilroberta-base-climate-f | 512 | 85.20%

(a) roberta-base

(b) bert-base-uncased

(c) distilroberta-base-climate-f

Figure 2. Learning curve showing variation in training and validation loss
with epochs ($\alpha$=5e-5)

(a) roberta-base

(b) bert-base-uncased

(c) distilroberta-base-climate-f

Figure 3. Learning curve showing variation training and validation loss with
epochs ($\alpha$=5e-6)

##  5\. Results and Discussions

###  5.1. Evaluation method

We evaluate the approaches detailed in the previous section by using the test
data comprising of 1859 samples of transaction texts. We use weighted F1 score
as the metric to compare the performance of the different approaches.

###  5.2. Comparison of training methods

##### Performance of zero-shot method

Table 2 shows the results from zero-shot approaches. We observe that some
useful insights can be drawn from the results. F1 score is in the range of
27-28% when using the title of commodity classes for computing the embeddings.
On the other hand, when the commodity description is used, the resulting F1
score is around 40-43 %. This makes sense, as descriptions provide more
details about the commodity classes than their titles. Therefore embeddings
computed using the descriptions are more meaningful and therefore result in a
better performance. We also see that all-mpnet-base-v2 performs better as
compared to the other 2 sentence transformers. This is also in accordance with
the details provided in documentation of the SBERT sentence transformers
222www.sbert.net/docs/pretrained_models.html. However, we see that the
performance of the text similarity approach has an overall low performance
with a low F1 score, highlighting the importance of a supervised approach for
this task.

##### Performance of Supervised Learning using classical models

: Table 3 shows the performance of classical modeling using the TF-IDF and
Word2Vec based vectorization of input. We see that Word2Vec has a better
performance as compared to TF-IDF. This makes sense because Word2Vec captures
the semantic relationships between words by representing them as dense
vectors. This allows it to capture contextual and relational information that
may not be captured by TF-IDF, which only considers term frequencies and
inverse document frequencies.
We also see that overall this approach has a strong improvement over the zero
shot classification approach. This can be due to the fact that Large Language
models such as sentence transformers are pre-trained mainly on texts in the
form of paragraphs and sentences. However in our problem, we see that the
transaction data are often not in the form of complete sentences, but in the
form of sentence fragments such as Computer and Peripheral Equipment expense,
or Farm crop packaging materials cost. Hence LLMs are not able to generalize
over these inputs in a zero-shot fashion and have a poor performance.

Figure 4. Commodity classes Spend and its Scope 3 emissions

##### Performance of Supervised Fine Tuning

Table 4 shows the performance of roberta-base, bert-base-uncased and
distilroberta-base-climate-f for different max_length, for a learning rate of
5e-5. We see that the performance of the models is not impacted by varying
max_length. This is because the number of tokens in input texts are in a much
smaller range. Hence, there was no truncation of the input text, which could
have impacted the model’s performance. We also observe that the performance of
ClimateBERT model is lower as compared to the other 2 models, which could be
due to the fact that it has fewer layers and parameters as compared to the
other two models.

Fig 2 shows the variation in training and validation loss with epochs for
max_length of 512, for the 3 LLM’s used in the analysis. We observe that in
all the 3 learning curves, the validation loss decreases sharply in the
initial epochs and then starts to slightly increase after around the 10th
epoch, while the training curve is almost constant. We repeated the training
after lowering the learning rate from 5e-5 to 5e-6. Figure 3 shows the
learning curve for the 3 models fine-tuned using the lower leaning rate. We
observe that fine-tuning using the lower learning rate resulted in a smoother
convergence and helped in achieving a better validation loss of the best
checkpoint model. For example, in roberta-base, the validation loss dropped
from 0.577 to 0.557, denoting a better fine-tuning. This also improved the F1
score on the test data for all the models, as can be seen from Table 5.

We also see that this approach outperforms the classical ML model using the
TF-IDF and Word2Vec based vectorization of input. The drastic improvement in
performance can be attributed to the fact that LLM’s have already been pre-
trained on a large corpus of texts. This helps the model in developing a
semantic understanding and learning of contextualized word representations.
Fine-tuning these models on the input data allows the model to adapt to the
task specific patterns, features, and labels, refining their representations
and predictions for more accurate classification.

###  5.3. Ablation Study

##### Subset of training data

We experimented with the supervised learning approaches mentioned in Sections
4.2 and 4.3 using 50% of the training and validation data. Table 6 shows the
results from the different models. We see that there is a reduction in F1
score, when compared to the results from tables 3 and 4. This denotes that the
reduced training data may not be sufficient for the model to learn
effectively. With a smaller dataset, the model may struggle to capture the
full range of patterns and variations present in the financial ledger
descriptions, leading to decreased performance on test data. It also suggests
that having an even larger training dataset may further help improve the
model’s performance.

Table 6. Results for 50% smaller training data  Model | max_length | F1 (test)
---|---|---
TF-IDF |  | 64%
Word2Vec |  | 66%
roberta-base | 512 | 81.29%
bert-base-uncased | 512 | 82.18%
distilroberta-base-climate-f | 512 | 80.03%

###  5.4. Estimation and analysis of Scope 3 emission

Once the ledger descriptions are mapped into EEIO summary commodity classes,
we use EEIO emission factors (emission per $ spend) provided in Supply Chain
GHG Emission Factors for US Commodities and Industries v1.1 (sup, 2022) to
compute carbon footprint related to the expenses.

Figure 4 shows the sample expense in USD and its associated emission
distribution for various commodity classes for the considered use case. It is
interesting to see that the correlation between the expense and the respective
emissions across the commodity classes are not trivial and very counter-
intuitive. Some commodity classes have high expenditure with lower emission
and vice-versa. For example, the commodity classes like Administration,
Cleaning, General building, Professional services, etc., have relatively lower
emissions when compared to their expenses. Whereas the commodity Utilities has
relatively very high emission with respect to its expenditure. Similarly the
emissions of few commodities like Conveyance Systems, Waste, etc., are
comparably varying with the expenses. These detailed insights provides an
opportunity for the enterprises to take appropriate climate actions for
Scope3.

##  6\. Conclusion

In this paper, we proposed a first-of-a-kind framework for estimating Scope 3
emission leveraging large language models by classifying transaction/ledger
description of purchase good and services into US EPA EEIO commodity classes.
We have conducted extensive experiments with multiple state of the art large
language models (roberta-base, bert-base-uncased and distilroberta-base-
climate-f) as well as non-foundation classical approaches (TF-IDF and Word2Vec
) and found that domain adapted foundation models are outperforming the
classical approaches. We also see that supervised fine-tuned foundation model
approach has a strong improvement over the zero shot classification approach.
While there is not significant performance variation across the chosen
foundation models, we see roberta-base performs slightly better than others
for the considered use case. We further conducted experiments with
hyperparameter tuning (learning rate and maximum sequence length) for the
foundation models, and found that model performance is not sensitive to
maximum sequence length but the lower learning rate provides better
convergence characteristics. The experimental results show that the domain
adapted foundation model performs as good as domain-expert classification, and
even outperforms domain-expert in many instances. As financial transactions
are easily accessible, the proposed method will certainly help to accelerate
the Scope 3 calculation at an enterprise scale and will help to take
appropriate climate actions to achieve SDG 13.

## References

  * (1)
  * sup (2022) 2022\.  Supply Chain GHG Emission Factors for US Commodities and Industries v1.1.  (2022).  https://doi.org/10.23719/1524524
  * Balaji et al. (2023) Bharathan Balaji, Venkata Sai Gargeya Vunnava, Geoffrey Guest, and Jared Kramer. 2023.  CaML: Carbon Footprinting of Household Products with Zero-Shot Semantic Text Similarity. In _Proceedings of the ACM Web Conference 2023_. 4004–4014.
  * Corringham et al. ([n. d.]) Tom Corringham, Daniel Spokoyny, Eric Xiao, Christopher Cha, Colin Lemarchand, Mandeep Syal, Ethan Olson, and Alexander Gershunov. [n. d.].  BERT Classification of Paris Agreement Climate Action Plans.  ([n. d.]).
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171–4186.  https://doi.org/10.18653/v1/n19-1423
  * Farsan et al. (2018) Alexander Farsan, Andres Chang, Annemarie Kerkhof, B Cserna, Chendan Yan, Fernando Rangel Villasana, Nicole Labutong, B Ketelaar, D Van Petersen, F De Haan, et al. 2018\.  Best practices in scope 3 greenhouse gas management.
  * Garrido-Merchán et al. (2023) Eduardo C Garrido-Merchán, Cristina González-Barthe, and María Coronado Vaca. 2023.  Fine-tuning ClimateBert transformer with ClimaText for the disclosure analysis of climate-related financial risks.  _arXiv preprint arXiv:2303.13373_ (2023).
  * Lacoste et al. (2021) Alexandre Lacoste, Evan David Sherwin, Hannah Kerner, Hamed Alemohammad, Björn Lütjens, Jeremy Irvin, David Dao, Alex Chang, Mehmet Gunturkun, Alexandre Drouin, et al. 2021\.  Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark.  _arXiv preprint arXiv:2112.00570_ (2021).
  * Leontief (1970) Wassily Leontief. 1970\.  Environmental Repercussions and the Economic Structure: An Input-Output Approach.  _The Review of Economics and Statistics_ 52, 3 (1970), 262–271.  http://www.jstor.org/stable/1926294
  * Li et al. (2019) Mo Li, Thomas Wiedmann, and Michalis Hadjikakou. 2019\.  Enabling full supply chain corporate responsibility: scope 3 emissions targets for ambitious climate change mitigation.  _Environmental science & technology_ 54, 1 (2019), 400–411.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _CoRR_ abs/1907.11692 (2019).  arXiv:1907.11692 http://arxiv.org/abs/1907.11692
  * Luccioni et al. (2020) Alexandra Luccioni, Emily Baylor, and Nicolas Duchene. 2020\.  Analyzing sustainability reports using natural language processing.  _arXiv preprint arXiv:2011.08073_ (2020).
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013\.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_ (2013).
  * Miller and Blair (2009) R. E. Miller and P. D Blair. 2009.  _Input-Output Analysis: Foundations and Extensions_.  Cambridge University Press, Cambridge.
  * Nations (2021) United Nations. 2021\.  The Sustainable Development Goal 13 : Take urgent action to combat climate change and its impacts.  https://unstats.un.org/sdgs/report/2021/Goal-13/
  * Nations (2022) United Nations. 2022\.  The Sustainable Development Goals Report.  https://unstats.un.org/sdgs/report/2022/The-Sustainable-Development-Goals-Report-2022.pdf
  * Nguyen et al. (2022) Quyen Nguyen, Ivan Diaz-Rainey, Adam Kitto, Ben McNeil, Nicholas Pittman, and Renzhu Zhang. 2022\.  Scope 3 Emissions: Data Quality and Machine Learning Prediction Accuracy.  (2022).
  * Nugent et al. (2021) Tim Nugent, Nicole Stelea, and Jochen L Leidner. 2021\.  Detecting Environmental, Social and Governance (ESG) Topics Using Domain-Specific Language Models and Data Augmentation. In _International Conference on Flexible Query Answering Systems_. Springer, 157–169.
  * Protocol et al. (2004) Greenhouse Gas Protocol, Greenhouse Gas Protocol Initiative, et al. 2004\.  A corporate accounting and reporting standard.  _World Resources Institute and World Business Council for Sustainable Development_ (2004).
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.  https://arxiv.org/abs/1908.10084
  * Schmidt et al. (2022) Mario Schmidt, Moritz Nill, and Johannes Scholz. 2022\.  Determining the Scope 3 Emissions of Companies .  _Chemical Engineering & Technology_ 45, 7 (2022), 1218–1230.
  * Serafeim and Velez Caicedo (2022) George Serafeim and Gladys Velez Caicedo. 2022.  Machine Learning Models for Prediction of Scope 3 Carbon Emissions.  _Available at SSRN_ (2022).
  * Tracker (2022) Climate Action Tracker. 2022\.  2100 Warming projections.  https://climateactiontracker.org/global/temperatures/
  * Vigon et al. (2020) B.W. Vigon, D.A. Tolle, B.W. Cornaby, H.C. Latham, C.L. Harrison, T.L. Boguski, R.G. Hunt, and J.D. Sellers. 2020\.  _Life-Cycle Assessment: Inventory Guidelines and Principles_.
  * Vinuesa et al. (2020) Ricardo Vinuesa, Hossein Azizpour, Iolanda Leite, Madeline Balaam, Virginia Dignum, Sami Domisch, Anna Felländer, Simone Daniela Langhans, Max Tegmark, and Francesco Fuso Nerini. 2020.  The role of artificial intelligence in achieving the Sustainable Development Goals.  _Nature communications_ 11, 1 (2020), 233.
  * Webersinke et al. (2022) Nicolas Webersinke, Mathias Kraus, Julia Bingler, and Markus Leippold. 2022.  ClimateBERT: A Pretrained Language Model for Climate-Related Text. In _Proceedings of AAAI 2022 Fall Symposium: The Role of AI in Responding to Climate Challenges_.  https://doi.org/10.48550/arXiv.2212.13631
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020\.  Transformers: State-of-the-Art Natural Language Processing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_. Association for Computational Linguistics, Online, 38–45.  https://www.aclweb.org/anthology/2020.emnlp-demos.6


# Getting started: How a supersonic stellar wind is initiated from a
hydrostatic surface

Stan Owocki  Department of Physics & Astronomy, Bartol Research Institute,
University of Delaware, Newark, DE 19716 USA
email<EMAIL_ADDRESS>
(2022)

###### Abstract

Most of a star’s mass is bound in a hydrostatic equilibrium in which pressure
balances gravity. But if at some near-surface layer additional outward forces
overcome gravity, this can transition to a supersonic, outflowing wind, with
the sonic point, where the outward force cancels gravity, marking the division
between hydrostatic atmosphere and wind outflow. This talk will review general
issues with such transonic initiation of a stellar wind outflow, and how this
helps set the wind mass loss rate. The main discussion contrasts the flow
initiation in four prominent classes of steady-state winds: (1) the pressure-
driven coronal wind of the sun and other cool stars; (2) line-driven winds
from OB stars; (3) a two-stage initiation model for the much denser winds from
Wolf-Rayet (WR) stars; and (4) the slow “overflow” mass loss from highly
evolved giant stars. A follow on discussion briefly reviews eruptive mass
loss, with particular focus on the giant eruption of $\eta$ Carinae.

###### keywords:

Sun: solar wind; stars: early-type; stars: mass loss; stars: Wolf-Rayet;
stars: AGB.

††volume: 366††journal: The Origin of Outflows from Evolved Stars††editors:
Leen Decin& Clio Gielen, eds.

##  1 Introduction

Figure 1: Icons to represent analogies for processes initiating the four
different kinds of steady stellar wind outflow. Figure 2: Comparison of the
properties of solar and stellar steady wind outflows (left) with the example
of eruptive mass loss from $\eta$ Carinae (right center), and contrasting that
with the explosive mass ejection from a supernova. (right)

To set the stage for this symposium’s exploration of “The Origin of Outflows
from Evolved Stars”, I have been asked to review the basic processes
underlying the initiation of such outflows. Most of a star’s mass is bound in
a hydrostatic equilibrium for which the outward push of pressure balances the
inward pull of gravity. But if at some near-surface layer additional outward
forces overcome gravity, this can transition to a supersonic, outflowing wind,
with the sonic point, where the outward force cancels gravity, marking the
division between hydrostatic atmosphere and wind outflow. This summary reviews
general issues with such transonic initiation of a stellar wind outflow, and
how this helps set key wind properties like the mass loss rate and wind flow
speed.

As summarized in figure 1, much of the discussion, given in §2, focuses on
four distinct types of steady-state outflows, namely: (1) the pressure-driven
coronal wind of the sun and other cool stars; (2) line-driven winds from OB
stars; (3) a two-stage initiation model for the much denser winds from Wolf-
Rayet (WR) stars; and (4) the slow “overflow” mass loss from highly evolved
giant stars. A follow on discussion in §3 briefly reviews eruptive mass loss
(see figure 2), with particular focus on a binary merger model for giant
eruption of $\eta$ Carinae, perhaps the most famous of the class of eruptive
Luminous Blue Variables (eLBV).

Figure 3: Images of the solar disk. Left: White light showing solar
photosphere; Middle: X-rays showing magnetically confined hot coronal loops ;
Right: Solar eclipse showing how coronal loops are stretched outward into
pointed “helmet” streamers by the expansion into the solar wind

##  2 Initiation of steady winds

The mass $M(r)$ within a local radius $r$ of a spherically symmetric star
exerts an inward gravitational acceleration $g=GM(r)/r^{2}$, with $G$ the
gravitation constant; for local mass density $\rho$, this is balanced by the
acceleration associated with a pressure gradient $-(1/\rho)dP/dr$. Using the
ideal gas law to write the pressure $P=\rho kT/\mu$ in terms of temperature
$T$, molecular weight $\mu$, and Boltzmann’s constant $k$, we find the
pressure drops exponentially with a characteristic pressure scale height,

$H\equiv\frac{P}{|dP/dr|}=\frac{kT}{\mu g}=\frac{a^{2}}{v_{orb}^{2}}R\,.$ (1)

Here the last equality casts the value at surface radius $r=R$ in terms of
ratio of the (isothermal) sound speed $a\equiv\sqrt{kT/\mu}$ to surface
orbital speed $v_{orb}\equiv\sqrt{GM/R}$. For the solar photosphere, we find
$a\approx 10$ km/s while $v_{orb}\approx 420$ km/s, thus implying a scale
height that is a tiny fraction $H/R\approx 4\times 10^{-4}$ of the solar
radius. This is the essential reason the edge of the solar disk appear so
sharp in white light images, such as shown in the leftmost panel in figure 3.

Indeed, main sequence stars are all characterized by a similarly small ratio
$H/R$, implying that some other force must kick in to overcome gravity and
drive a wind outflow. In the luminous OB and WR stars, this stems from the
momentum of scattered radiation, as discussed in §§2.2 and 2.3.

However, for evolved giants, including Red Giants (RG), Red Super-Giants (RSG)
and Asymptotic Giant Branch (AGB) stars, the much weaker surface gravity
implies a less tiny ratio for $H/R\gtrsim 0.1$, making it easier for internal
variations to induce mass loss, as discussed in §2.4.

###  2.1 Pressure-driven coronal winds

The initial prototype for wind mass loss came from the realization by G.
Parker that the high (MK) temperature of the solar corona would lead to a
supersonic expansion we now know as the solar wind. For a characteristic
coronal temperature of 2 MK, we find the scale height ratio is now $H/R\approx
0.14$. The X-ray image in the middle panel of figure 3 shows how the hot
corona thus has a much greater extension above the solar surface and beyond
the solar limb.

If this coronal temperature is kept high – through extended heating and
outward thermal conduction –, the radial drop of gravity implies this ratio
increases outward; as such, the pressure no longer continues to drop
exponentially, but rather asymptotically approaches a finite value,
$P_{\infty}$, at large radii, $r\rightarrow\infty$. Relative to the initial
pressure $P_{o}$ at the coronal base, the total drop in pressure for a
hydrostatic, isothermal corona is given by

$\frac{P_{o}}{P_{\infty}}\approx
e^{R/H}~{}~{};~{}~{}\log\frac{P_{o}}{P_{\infty}}\approx\frac{6}{T/{\rm
MK}}\,.$ (2)

The latter equality shows the pressure drops by 6 decades for $T=1$ MK, and
only 3 decades for $T=2$ MK. By comparison, from the solar transition region
at the coronal base to the interstellar medium, the pressure drop is actually
much greater, $\log(P_{tr}/P_{ism})\approx 12$. The upshot is that an
extended, hot corona can not be maintained in hydrostatic equilibrium;
instead, as shown by the outward streamers from the eclipse image in figure 3,
it must undergo an outward, supersonic expansion known as the solar wind.

Figure 4: Schematic to illustrate energy input into the nearly hydrostatic,
subsonic coronal base vs. that into the supersonic wind above the Parker sonic
radius $R_{s}$. For the former, the net input vs. loss by conduction back into
the underlying atmosphere leads to coronal heating that sets the coronal
temperature and location and density at the sonic radius, thus fixing the
associated wind mass loss rate. For the latter, any further energy addition
increases the wind flow speed, with the wind expansion providing the primary
mechanism to carry out the total net amount of coronal heating.

As illustrated by the upper left panel of figure 1, this solar wind expansion
can be thought of as analogous to the release valve of a pressure cooker,
driven fundamentally by mechanical heating generated by magnetic turbulence in
the underlying solar atmosphere. As illustrated in figure 4, some of this
upward energy flux is lost back to the solar atmosphere through thermal
conduction, but the net effect leads to a thermal runaway that raises the
coronal temperature to millions of Kelvin, several hundreds times higher than
the photospheric temperature $T\approx 5800$ K.

To account for the advective acceleration $v(dv/dr)$ in a spherically
expanding wind, one can use mass continuity to split the pressure gradient
force into terms that scale with this acceleration and the sphericity,
yielding a steady-state equation of motion,

$\left(v-\frac{a^{2}}{v}\right)\frac{dv}{dr}=-\frac{GM}{r^{2}}+\frac{2a^{2}}{r}\,.$
(3)

In the subsonic region $v\ll a$, this reduces to the condition for hydrostatic
equiliibrium; but at a critical (“Parker”) radius,

$R_{s}=\frac{GM_{\odot}}{2a^{2}}=\frac{R_{\odot}^{2}}{2H}=3.5R_{\odot}\left(\frac{2{\rm
MK}}{T}\right)\,,$ (4)

where the RHS vanishes, the hydrostatic coronal base transitions to a
supersonic outflow, driven by the high gas pressure associated with the high
coronal temperature.

The density and radius of this sonic point set the wind mass loss rate
${\dot{M}}=4\pi\rho_{s}aR_{s}^{2}$, with typical values ${\dot{M}}\approx
10^{-14}M_{\odot}$/yr. Physically, this is set by the level and location of
the coronal heating. Heat added within the subsonic coronal base increases the
scale height, with less drop in density toward a closer sonic point, and thus
a direct increase in ${\dot{M}}$. In contrast, extended heating into the
supersonic wind, where the mass loss rate is already fixed, instead leads to
higher energy per unit mass, and thus a higher wind speed.

While the detailed mechanisms for coronal heating remain a subject of much
current research, the thermal runaway and associated coronal expansion are
thought to be quite robust consequences of the turbulence generated in all
cool stars ($T_{\ast}<10,000$ K) with convective envelopes associated with the
opacity blockage from Hydrogen recombination. The low coronal density needed
to avoid strong radiative cooling limits the mass loss rates to values, of
order $10^{-14}M_{\odot}$/yr, that do not appreciably reduce the stellar mass
over evolutionary timescales. But the enhanced angular momentum loss
associated with a global magnetic field can lead to an effective spindown of
the stellar rotation.

###  2.2 Line-driven winds from OB stars

In more massive, hotter OB stars with surface temperatures $T_{\ast}>10,000$ K
hydrogen remains ionized up to the surface; such hot stars thus lack the
H-recombination convection zones and associated the magnetic turbulence that
heats the hot corona, and associated pressure-driven expansion of cool star
winds. However such hot stars have a much higher radiative luminosity, and the
outward force from line scattering of this radiation can overcome gravity and
so drive a line-driven stellar wind outflow. For opacity $\kappa_{\nu}$ at a
frequency $\nu$ with radiative flux $F_{\nu}$, the total radiative
acceleration depends on the frequency integral,

$g_{rad}=\int_{0}^{\infty}d\nu\,\frac{\kappa_{\nu}F_{\nu}}{c}=\frac{\kappa_{e}F}{c}\,$
(5)

where the last equality applies in the simplest case of continuum scattering
by free electrons, with $F$ the bolometric flux.

For a fully ionized gas with solar hydrogen mass fraction $X=0.72$, such
electron scattering has an opacity $\kappa_{e}=0.34\,{\rm cm}^{2}$/g. The
ratio of the associated radiative acceleration to gravity is given by the
Eddington parameter,

$\Gamma_{e}\equiv\frac{\kappa_{e}F/c}{g}=\frac{\kappa_{e}L}{4\pi
GMc}\approx\frac{M}{200M_{\odot}}\,,$ (6)

wherein the inverse-square radial dependence of both the radiative flux
$F=L/4\pi r^{2}$ and gravity $g=GM/r^{2}$ cancels, showing this Eddington
parameter depends only on the ratio $L/M$ of luminosity to mass. The last
equality applies for the standard radiative envelope scaling $L\sim M^{3}$,
and provides a rationale for the upper limit to stellar mass, which is
empirically found to be around $200M_{\odot}$. Stars that approach or exceed
the Eddington limit $\Gamma_{e}=1$ can have strong eruptive mass loss, as
discussed in §3 for eruptive Luminous Blue Variable stars like $\eta$ Carinae.

Figure 5: Left: Illustration of resonant scattering between two bound levels
of an ion. Middle: The doppler-shifted desaturation that spreads line
absorption over a large range of frequencies, ranging up to the frequency
associated with the doppler shift from the wind terminal speed, $v_{\infty}$.
Right: Continuum photons propagating to the doppler-shifted line resonance
that is thermally broadened by over a Sobolev length $l_{sob}=v_{th}/(dv/dr)$,
where $v_{th}$ is the ion thermal speed.

But the steady winds from luminous stars with $\Gamma_{e}\gtrsim 10^{-3}$ are
understood to be driven by the line scattering from electrons bound into heavy
ions ranging from CNO to Fe and Ni. As illustrated in the leftmost panel of
figure 5, the resonance nature of such bound-bound line-scattering greatly
enhances the opacity, by a factor ${\bar{Q}}\gtrsim 10^{3}$ ([Gayley(1995),
Gayley 1995]), thus making it possible to overcome gravity and drive a wind
outflow for stars with $\Gamma_{e}\gtrsim 1/{\bar{Q}}\approx 10^{-3}$.

In practice this maximal line acceleration from optically thin scattering is
reduced by the saturation of the reduced flux within an optically thick line.
But as illustrated in the middle panel of figure 5, the doppler shift
associated with the wind acceleration acts to desaturate this line absorption,
effectively sweeping the absorption through a broad frequency band extending
out to the frequency associated with the doppler shift from the wind terminal
speed, $v_{\infty}$.

The right panel of figure 5 illustrates how the wind doppler shift of line
resonance concentrates the interaction of continuum photons into a narrow
resonance layer with width set by the Sobolev length, $\ell\equiv
v_{th}/(dv/dr)$ ([Sobolev(1960), Sobolev(1960)]), associated with acceleration
through the ion thermal speed $v_{th}$ that broadens the line profile. In the
outer wind where $\ell\ll r$, the line acceleration for optically thick lines
is reduced by $1/\tau$, where the Sobolev optical depth
$\tau\equiv{\bar{Q}}\kappa_{e}\rho\ell$, giving then a line acceleration
$\Gamma_{thick}\sim(1/\rho)(dv/dr)$ that itself scales with the wind
acceleration. Within this Sobolev approximation, [CAKI75, Castor, Abbott and
Klein (1975; hereafter CAK)] developed a formalism that accounts for the
radiative acceleration from an ensemble of thick and thin lines, deriving
thereby scalings for the wind speeds and associated wind mass loss rate. Much
as in the solar wind, the terminal wind speeds scale with the surface escape
speed $v_{esc}\equiv\sqrt{2GM/R}$, with values up to $v_{\infty}\approx 2000$
km/s. However, the mass loss rates can range up to ${\dot{M}}\sim
10^{-5}M_{\odot}$/yr, and so up to a billion times that of the solar wind!

Figure 6: Alternative rendition of doppler shift from wind acceleration, now
depicted as redshift of stellar photons until the come into resonance with a
thermally broadened line. This allows depiction of the forward and backward
scattering of photons escaping this layer. In the outer, supersonic wind of
the right panel, this is fore-aft symmetric, canceling any net recoil from the
diffuse radiation. But for inner, transonic wind of the left panel, there can
be a net asymmetry between the forward/background scattering, leading then to
a non-zero diffuse line force. The consequences of this are illustrated in
figure 7 and discussed in the text.

A longstanding issue for this Sobolev-based CAK model regards the wind
initiation near wind sonic point. The effective desaturation of a Sobolev
model requires,

$\frac{v_{th}}{dv/dr}\equiv\ell\ll
H\approx\frac{\rho}{|d\rho/dr|}\approx\frac{v}{dv/dr}\,.$ (7)

At the sonic point $v=a$, this requires $v_{th}\ll a$. Fortunately, because
line driving is by heavy ions with $v_{th}/a\approx 0.3$, this condition is
marginally satisfied; but it does indicate that a more careful treatment is
warranted for the radiative transfer and line driving in this transonic
region.

For this it is important to recognize that, as illustrated in figure 6, line-
transfer actually occurs mainly via scattering not pure absorption. The right
panel shows that, in the highly supersonic outer winds, the escape of photons
from the Sobolev resonance is nearly fore-aft symmetric, canceling their
recoil, so that the net line-force is the same as if the photon had been
purely absorbed. The left panel shows, however, that in the inner, transonic
region this scattering can become asymmetric, with a net recoil and so a non-
zero diffuse line-force.

Figure 7: Radiative acceleration with scattering in the transonic regino
(left) and the result initiation of wind velocity, for both Smooth Source
Function (SSF; top) and Escape Integral Source Function (EISF; bottom) models
of the scattering line transfer.

Figure 7 compares the radiative driving (left) and resulting wind velocity
(right) for two different approximations for the transonic scattering line
transfer ([Owocki & Puls(1996), Owocki & Puls(1999), Owocki & Puls 1996,
1999]). The top row shows a Smooth Source Functon (SSF) model, which assumes
coupling with the continuum can keep the line source function nearly constant
through the sonic region; the difference in fore/aft escape leads to a
positive diffuse line force, which effectively compensates for the reduction
in direct line force associated with the incomplete desaturation in this
transonic region. The net result is that the wind driving and velocity are
very similar to the standard CAK model.

The bottom row shows an Escape Integral Source Function (EISF) model,
applicable for lower-density cases without sufficient continuum coupling to
keep the source function smooth in the transonic region. The greater escape
from the increasing velocity gradient leads now to a marked dip in the source
function, with a weaker or even inward diffuse line force that further reduces
the net line driving the subsonic region, and thus now a sharp, step-like jump
in wind velocity around the sonic point.

An overall point is that in all these models the onset of line-driving near
the sonic point represents an effective line-driven suction, which draws up
mass from the underlying hydrostatic equilibrium of the subsonic region. The
reduction in pressure from the outer line-driving induces the underlying
subsonic region to expand upward, in much the way that, as illustrated in
figure 1, the suction on a straw draws up liquid from a glass. This outside-in
suction contrasts with the inside-out thermal expansion of a pressure cooker,
and of the analogous gas pressure-driven solar wind.

###  2.3 Iron-bump initiation of WR winds

For the much higher mass loss rates of Wolf-Rayet stars, the winds can
themselves become optically thick, with the overall wind blanketing increasing
the temperature of the transonic region well above the stellar effective
temperature, so bringing it closer to the temperature $T\approx 200,000$ K for
a strong opacity bump associated with the huge number of overlapping iron
lines. In OB stars, this “iron opacity bump” induces a narrow sub-layer of
convection that generates gravity waves and the associated surface macro-
turbulence inferred from broadening of photospheric spectral lines. But in WR
stars close to the Eddington limit, the inefficiency of the more near-surface
convection can instead cause the net force to exceed the Eddington limit,
leading to either an inflation of the stellar envelope ([Gräfener et
al.(2012), Graefener et al. 2012]), or even initiation of an outflow
([Poniatowski et al.(2021), Poniatowski et al. 2021]).

Figure 8:  Left: Gravitationally scaled flow energy $w=v^{2}/v_{esc}^{2}$
plotted vs. scaled radius $x=1-R_{c}/r$, where $R_{c}$ is the core radius near
the wind sonic point. The various curves illustrate failed outflow solutions
initiated at various densities by the iron opacity bump, with thus the various
labeled mass loss rates, ${\dot{M}}$, in $M_{\odot}$/yr. Right: Flow speed vs.
scale radius $x$ for two-stage wind acceleration model. Gray shading shows the
optically thick wind region where iron-bump opacity initiates a wind outflow,
which is sustained by including velocity desaturation that transitions to a
CAK-like line-driven outflow, as fitted in the dashed plus-sign curve by a
standard wind velocity law. This two-stage process can be characterized as a
kind of “dynamical inflation” of the outer stellar envelope. Figures adopted
from [Poniatowski et al.(2021), Poniatowski et al. (2021)].

As illustrated in the left panel of figure 8, the decline in this iron bump
opacity with decreasing temperature means that it alone cannot sustain such
outflow driving in the cooler, outer layers, leading then to a gravitational
slowing and eventual stagnation of the wind outflow. The right panel shows,
however, that including the velocity desaturation of CAK-like line-driving can
rekindle the outward acceleration and so sustain the outflow.

The overall two-stage model represents a kind of “dynamical inflation”, with
associated mass loss rates an order of magnitude higher than obtained in
O-stars of comparable luminosity, in extreme cases approaching ${\dot{M}}\sim
10^{-4}M_{\odot}$/yr. As illustrated in figure 1, it is somewhat akin to a
water bubbler stream that is then taken up by the suction from a drinker.
Talks by N. Moens and A. Sander in these proceedings discuss further such WR
wind acceleration.

Figure 9: Illustration of large convective plumes of a giant star, which can
help initiate overflows mass loss with high mass loss rate ${\dot{M}}$ but low
flow speeds $v_{\infty}$.

###  2.4 Slow overflow mass loss from giant stars

Finally let us turn to the strong mass loss that can occur from evolved giant
stars like Red (Super)Giants (RG and RSG) and those on the Asymptotic Giant
Branch (AGB). Like the overflowing glass illustrated in figure 1, this can be
thought of a kind spillage, with overflowing material escaping the much weaker
gravity of such stars, with flow speeds that barely exceed the reduced escape
speed. Even for the lower sound speed associated with a lower surface
temperature $T_{\ast}\sim 3000$ K, the larger radius now can lead to a
moderate scale height to radius ratio

$\frac{H}{R}=0.2\frac{T}{3000{\rm
K}}\,\frac{R/1000\,R_{\odot}}{M/M_{\odot}}\,,$ (8)

Fgure 8 shows how the associated convection cells in such stars can thus
extend over a substantial fraction of a stellar radius. As discussed in the
contribution by S. Hoefner in these proceedings, detailed 3D simulations show
this can sporadically suspend material to a large enough radius that the
temperature becomes low enough ($T\lesssim 1500$ K) to initiate dust
formation. The high opacity of this dust can lead to a radiative acceleration
that exceeds gravity, so driving material to full escape from the star.

For dust grains of radius $a$, density $\rho_{d}$ and mass fraction $X_{d}$,
the dust opacity can be written as

$\kappa_{d}=\frac{3X_{d}}{4a\rho_{d}}\approx
150\frac{cm^{2}}{g}\,\frac{X_{d}}{X_{d\odot}}\,\frac{0.1\mu
m}{a}\,\frac{1\,g/cm^{3}}{\rho_{d}}\,,$ (9)

where $X_{d\odot}=0.002$ is the maximum dust fraction for solar abundance of
dust-forming elements.

By comparison, the critical (Eddington) opacity to overcome gravity has the
scaling ([Höfner & Olofsson(2018), Hoefner and Olofsson 2018]),

$\kappa_{crit}=\frac{4\pi GMc}{L}\approx
2.6\frac{cm^{2}}{g}\,\frac{M}{M_{\odot}}\,\frac{5000L_{\odot}}{L}\,.$ (10)

Comparison shows that for luminous giant stars even partial dust formation
should have an opacity that is sufficient to drive material to escape.

There remains some debate as to whether dust driving is essential to giant-
star mass loss, or simply augments it. For example, there are models ([Kee et
al.(2021), e.g., Kee et al. 2021]) for steady outflows driven by turbulent
pressure, wherein as in pressure-driven coronal models, the mass loss rate is
set by the density at the extended sonic point, but with values that greatly
exceeds values of the solar wind, as high as $10^{-6}M_{\odot}$/yr. The
presentation by J. Sundqvist in these proceedings provides further discussion
of such turbulence-driven wind models.

##  3 Eruptive mass loss

###  3.1 Energy requirement for eLBV mass ejection

To complement the above discussion of the initiation of steady winds, let’s
next review the processes for initiating mass loss in an observational class
of luminous, massive stars known as “eruptive Luminous Blue Variables”
(eLBV’s). Figure 2 summarizes how these have properties that are intermediate
between steady solar and stellar winds and the explosive mass ejection of
core-collapse supernovae (SNe). In the latter, the energy generated by core
collapse deposits sufficient energy into the overlying stellar envelope to
blow it completely from star, over an initial dynamical timescale of seconds,
and with ejecta speeds that can approach $0.1$c.

Such SNe are thus at opposite extreme of the gradual, steady mass loss in
winds, for which initiation occurs in surface layers when an outward force
overcomes the inward gravitational acceleration. Even in the extreme cases of
WR winds or mass loss from giant stars, the energy needed to escape is
generally a small fraction of the available luminosity emitted by radiation.

The giant eruptiions seen from eLBV’s have properties intermediate between
winds and SNe. While their initiation may be sudden, their evolution can
extend over years or even decades, much longer than a dynamical timescale, but
generally short compared with a thermal relaxation timescale of the erupting
star. The mass fraction ejected can be up to 10-20% of the stellar, much less
than the full envelope ejection of SNe, but much larger than even the
cumulative mass loss of stellar winds. And unlike the fixed terminal speed of
winds, eLBV ejecta speed can have broad range, extending from a faster, low
density leading edge to a bulk of mass that just barely escapes the star’s
gravity.

A promising paradigm is to consider such eLBV eruptions as arising from a
quite sudden addition energy to the star’s envelope, which, unlike the
explosive addition of SNe, is however only some fraction $f<1$ of the envelope
binding energy.

Figure 10:  Left:Contours of Bernouli energy $B$ as a function of mass below
surface and time since addition of an energy that is half the star’s binding
energy over the outer $25M_{\odot}$. The resulting direct and reflected shocks
heat the outer layers, making the outer $\sim 7M_{\odot}$ become unbound with
$B>0$. Right: The resulting outflow of the stellar envelope, with escape of
$\sim 7.3M_{\odot}$, with the rest of the material falling back onto the star.
Figures adopted from [Owocki et al.(2019), Owocki et al. (2019)].

Figure 10 show results for 1D hydrodynamical simulations of the response when
a energy that is a fraction $f=1/2$ of the stellar binding energy is addied to
the outer 25 $M_{\odot}$ of a 100 $M_{\odot}$ star, as indicated by the arrow
along the left axis of the left panel. In terms of fluid parcels defined by
their mass coordinate from the surface, the contours show the time response of
the total “Bernouli” specific energy,

$B\equiv\frac{v^{2}}{2}-\Phi_{grav}+h\,,$ (11)

where $v$ is the flow speed, $\Phi_{grav}$ is the gravitational potential, and
$h$ is the total specific enthalpy from gas and radiation; in terms of gas
density and gas and radiation pressure, this is given by

$h=\frac{\frac{5}{2}P_{gas}+4P_{rad}}{\rho}=\frac{5}{2}\frac{kT}{\mu}+\frac{4a_{rad}T^{4}}{3\rho}\,,$
(12)

with $a_{rad}$ is the radiation constant. Note how the initial energy addition
induces a pair of direct and shock front that propagate toward the surface,
heating the gas there so that the increased enthalpy makes the total Bernouli
energy positive for the upper $\sim 7M_{\odot}$ from the surface. The result
is an outward expansion of the stellar envelope, with the positive energy mass
parcels with $M<7.4M_{\odot}$ escaping completely from the star, while the
negative energy mass parcels with $M>7.5M_{\odot}$ eventually back onto the
star.

The ejecta’s variations in time $t$ and radius $r$ for the velocity $v$,
density $\rho$, and temperature $T$ are quite well fit by similarity forms in
the variable $r/t\approx v$. Specifically the scaled density follows a simple
exponential decline $\rho t^{3}\sim\exp(-r/v_{o}t)$. This exponential
similarity leads to analytic scaling relations for total ejecta mass $\Delta
M$ and kinetic energy $\Delta K$ that agree well with the hydrodynamical
simulations, with the specific-energy-averaged speed related to the
exponential scale speed $v_{o}$ through ${\bar{v}}=\sqrt{2\Delta K/\Delta
M}=\sqrt{12}v_{o}$, and a value comparable to the star’s surface escape speed,
$v_{esc}$.

Unlike the fixed terminal speed $v_{\infty}$ of a stellar wind, a small amount
of material can be ejected at very high speeds, $>5000$ km/s. But like stellar
winds, gravity still plays a central role in controlling the mass and speed of
the outflow, through the ratio of the added energy to the gravitational
binding energy. This is distinct from standard SNe explosion, for which the
added energy essentially overwhelms the gravitational binding of the envelope,
leading to explosion speeds of order 0.1c

Figure 11: Schematic outline of multi-step process leading to a binary merger
that triggers the giant eruption of $\eta$ Carinae, with the post-eruption,
bipolar , super-Eddington wind shaping the Homunculus nebula. Figure adopted
from [Hirai et al.(2021), Hirai et al. (2021)].

###  3.2 Merger model for $\eta$ Carinae

The 1840’s giant eruption of $\eta$ Carinae is perhaps the most prominent and
extreme example of an eLBV, with the estimated $10-20\,M_{\odot}$ ejected mass
forming the bipolar ‘Homunculs’ nebula (shown in the 3rd image in figure 2).
Two key challenges are to understand both the energy source powering the
eruption, and the causes of the bipolar form. $\eta$ Carinae is known to have
a massive companion in an eccentric ($\epsilon\approx 0.9$ orbit with period
$\sim 5.5$ yr.

Figure 11 summarizes a recently proposed model by [Hirai et al.(2021), Hirai
et al. (2021)], in which an original triple system (phase 1) becomes unstable
due to mass exchange (phase 2), leading to swaps and close encounters that
result in a series of random ejecta that today are observed with source times
extending back several centuries. Eventually, orbital decay of the innermost
pair leads to a merger (phase 3), powering the 1840’s giant eruption. The
enhanced luminosity and rapid rotation of the post-merger star (phase 4)
drives a strong, bipolar, super-Eddington wind that sculpts the compressed
Homonculus nebula seen today.

##  4 Summary

Within this symposium on “The Origin of Outflows from Evolved Stars”, a
general overall theme for the above review on the initiation of such outflows
is the key role played by gravity, and the need for forces that can overcome
its inward pull to start an outflow, but also for this to be sustained to
allow escape from the star’s gravitational potential.

For coronal winds of the sun and other cool stars with convective envelopes,
this is achieved by the high gas pressure associated with turbulent heating
and thermal runway to temperatures, for which the gas internal energy becomes
comparable to the gravitational binding.

For hot, massive OB stars, the momentum of the high stellar luminosity becomes
effective tapped by the line-scattering for bound-bound transitions of heavy
ions, with the resonantly enhanced opacity maintained by desaturation of the
doppler shifted outflow; this results in a sudden onset of the line-force near
the sonic point, resulting in a line-driven suction of material from the star,
for which the details depend on the nature of scattering in this transonic
region.

In the much stronger, optically thick winds of Wolf-Rayet stars, the wind
blanketing increases the base temperature to be close enough to that of the
iron opacity bump, allowing a stronger wind initiation that is then sustained
in the outer regions by the line-opacity desaturation.

In giant stars, the reduction in surface gravity and escape speed allow for
giant convection cells to suspend material to levels with temperatures
$T\lesssim 1500$ K, cool enough to initiate dust formation; the coupling of
this enhanced dust opacity to the high luminosity can then dirve material to
full escape from the star. But even without dust formation, the turbulent
pressure can drive significant mass outflows against the weak gravity.

For eLBV eruptions, a key again is to provide an energy deposition to allow
some fraction of the stellar envelope to gain sufficient energy to be ejected
from the star. Instead of a steady wind, each mass parcel follows a trajectory
tied to its total energy, with now a distribution of escape speeds that are
described by a similarity form, with fall back to the star for bulk of
material with net negative total energy.

While the outline here presents an idealized view of steady vs. impulsive mass
loss, in practice real outflows will exhibit combinations of such traits, e.g.
with infall in some WR winds, and with eLBV eruptions punctuated by quasi-
steady super-Eddington winds. The further contributions in these proceeding
explore in greater detail the many variations on these themes that occur in
the outflows from evolved stars.

## Acknowledgments

I thank the IAUS366 SOC, and particularly its chair Leen Decin, for the
invitation to give this opening review. I also thank L. Poniatowski and R.
Hirai for permission to use figures 8 and 11, respectively.

## References

  * [Castor et al.(1975)] Castor, J. I., Abbott, D. C., & Klein, R. I. 1975, ApJ.
  * [Gayley(1995)] Gayley, K. G. 1995, ApJ, 454, 410.
  * [Gräfener et al.(2012)] Gräfener, G., Owocki, S. P., & Vink, J. S. 2012, A&A, 538, A40.
  * [Hirai et al.(2021)] Hirai, R., Podsiadlowski, P., Owocki, S. P., et al. 2021, MNRAS, 503, 4276.
  * [Höfner & Olofsson(2018)] Höfner, S. & Olofsson, H. 2018, A&A Rev., 26, 1.
  * [Kee et al.(2021)] Kee, N. D., Sundqvist, J. O., Decin, L., et al. 2021, A&A, 646, A180.
  * [Owocki & Puls(1996)] Owocki, S. P. & Puls, J. 1996, ApJ. 462, 894.
  * [Owocki & Puls(1999)] Owocki, S. P. & Puls, J. 1999, ApJ, 510, 355.
  * [Owocki et al.(2019)] Owocki, S. P., Hirai, R., Podsiadlowski, P., et al. 2019, MNRAS, 485, 988.
  * [Poniatowski et al.(2021)] Poniatowski, L. G., Sundqvist, J. O., Kee, N. D., et al. 2021, A&A, 647, A151.
  * [Sobolev(1960)] Sobolev, V. V. 1960, Cambridge: Harvard University Press, 1960.

DecinIn your discussion of eruptive mass loss, what fraction of the stellar
mass can be lost in such eruptions?

Owocki It depends on both the location of the energy addition and its fraction
of the stellar binding energy. The specific model I discussed ejected about 7%
of the mass of the star’s 100$M_{\odot}$, but a parameter study shows it is
possible to eject more or less than this. A large energy added in the stellar
core can eject the entire stellar envelope, as in a SNe explosion. Energy
addition to a near-surface layer will at most eject the mass in that layer.
Energy from a merger seems sufficient to eject $>10$% of a star’s mass, as
inferred for $\eta$ Carinae.

MellahI had a question about the bipolar outflow, with the polar outflow being
faster and more dilute

OwockiNo, it’s not more dilute, because in a rapidly rotating radiative
envelope, the bright poles can drive both a faster and denser outflow. In this
model, that’s how the Homunculus shape is formed.

MellahSo you wouid not see the material in the orbital plane.

OwockiWell, there is some material ejected mechanically in the
rotational/orbital plane, and this may be the origin of the observed
equatorial skirt.

De MarcoIn a binary, there’s of course the straight gravity of the companion
to help a star to lose mass. But do you envision a way in which a compansion
can trigger mass loss in a way that a star uses its own reservoir of energy?

Owocki Well there are some ideas, including I believe by yourself, for how
orbital motion can excite pulsation, with perhaps certain resonances,
resulting then in episodic mass ejection. But to get a giant eruption, where,
as in $\eta$ Carinae, the star loses 10% or more of its mass, I think you need
to add a large source of energy, either externally from a merger, or some kind
of ignition of enhanced burning in the core.

SahaiIn regards to mass loss in binaries, I wanted to point out there’s a
beautiful example in our Galaxy called the Boomerang nebula, wherein there’s a
huge amount of mass ejected from an intermediate mass star with relatively
high speeds. An other example is V Hydra, where one sees bullet-like ejection
triggered from periastron passage of a companion.

OwockiYes, I didn’t intend my talk to be comprehensive, and indeed the main
focus was on initiation of wind mass loss. But certainly the topic of eruptive
mass loss, and the role of binarity, is a broad area worthy of much further
discussion in this symposium.