Research Application H. Li, W. Kam-Kwai, C. Liu, Y. Luo, and H. Qu are with
Department of Computer Science and Engineering of HKUST. Email: {hliem,
kkwongar<EMAIL_ADDRESS><EMAIL_ADDRESS>and
<EMAIL_ADDRESS>J. Chen is with with School of Computer Science and
Technology of ECNU. Email<EMAIL_ADDRESS>Y. Zhang and Alexis K. H.
Lau are with Division of Environment and Sustainability of HKUST. Email:
<EMAIL_ADDRESS>and<EMAIL_ADDRESS>D. Liu is with Department of Computer
Science of UC Davis Email<EMAIL_ADDRESS>
# Save It for the “Hot” Day: An LLM-Empowered Visual Analytics System for Heat
Risk Management

Haobo Li  Wong Kam-Kwai  Yan Luo  Juntong Chen  Chengzhong Liu
Yaxuan Zhang  Alexis Kai Hon Lau  Huamin Qu  Dongyu Liu

###### Abstract

The escalating frequency and intensity of heat-related climate events,
particularly heatwaves, emphasize the pressing need for advanced heat risk
management strategies. Current approaches, primarily relying on numerical
models, face challenges in spatial-temporal resolution and in capturing the
dynamic interplay of environmental, social, and behavioral factors affecting
heat risks. This has led to difficulties in translating risk assessments into
effective mitigation actions. Recognizing these problems, we introduce a novel
approach leveraging the burgeoning capabilities of Large Language Models
(LLMs) to extract rich and contextual insights from news reports. We hence
propose an LLM-empowered visual analytics system, , that integrates the
precise, data-driven insights of numerical models with nuanced news report
information. This hybrid approach enables a more comprehensive assessment of
heat risks and better identification, assessment, and mitigation of heat-
related threats. The system incorporates novel visualization designs, such as
“thermoglyph” and news glyph, enhancing intuitive understanding and analysis
of heat risks. The integration of LLM-based techniques also enables advanced
information retrieval and semantic knowledge extraction that can be guided by
experts’ analytics needs. Our case studies on two cities that faced
significant heatwave events and interviews with five experts have demonstrated
the usefulness of our system in providing in-depth and actionable insights for
heat risk management.

###### keywords:

Heat risk management, climate change, numerical model, news data, large
language model, visual analytics

The interface of Havior (Heat Savior). The Meteorological Panel (A)
facilitates numerical understanding of meteorology, including temporal trends
(A1), temporal distribution (A2), and spatial distribution (A3). The
“thermoglyph” of Hong Kong (A4) intuitively shows the city-based pattern and
correlation between temperature and percentile. The News Panel (B) supports
human-in-the-loop news retrieval and enhancement in their semantic
understanding, in terms of topic-based hierarchies (B1) and risk-based
semantic proximity (B2) of retrieved news. The News List (B3) provides details
of structural information in the retrieved news on demand with supportive
visual cues. The Summary Panel (C) enables experts to examine the integration
of news and numeric city risk model (C1), pose contextual questions (C2), and
generate risk management reports.

Introduction

Extreme climate events [1], particularly those related to heat [27], have seen
a marked increase in intensity, frequency, and duration in recent years,
raising significant concerns globally. NASA has confirmed 2023 as the warmest
year on record. Heat risks are associated with excess mortality due to
temperatures above long-term averages during summer and specific extreme
events like heatwaves [13]. The adverse impacts extend beyond personal health,
as reduced workforce productivity is observed due to heat-related health
issues among employees [48]. They can further cause severe damage to critical
infrastructure, including buildings, power grids, roads, and rail lines,
leading to disruptions in daily life and economic activities [41].

Developing effective strategies for heat risk management, therefore, becomes
increasingly urgent. This typically involves three parts: identifying the
extreme climate event, assessing the associated risk levels, and determining
the actions to mitigate the adverse effects of such events [58, 32], such as
issuing a severe heat warning or enforcing heat-specific workplace standards.
However, this is not a trivial decision-making task as it requires a
comprehensive understanding of the complex interplay between various factors,
including meteorological, urbanization, demographic, and socioeconomic factors
[62, 13, 24].

To assess the risk levels of heat-related extreme events, numerical models are
dominantly used by environmental researchers [64, 69, 62]. However, they are
limited in three aspects. Firstly, their sparse spatial and temporal
resolution offers insufficient data-driven support for effective risk
management. For instance, the fifth-generation reanalysis (ERA5) data [22],
one of the most used domain data, yields only one estimate for an area
spanning approximately $27.75\times 27.75km^{2}$ per hour. This coarse-grained
resolution can only predict normalized results and overlook extreme
conditions. Secondly, these approaches aim to predict meteorological variables
(e.g., temperature and humidity), failing to capture the complex risk dynamics
involving human behaviors and social factors [53]. Thirdly, the preparedness
and response of society to the risk and the instructions for citizens are
absent.

Analyzing news articles about environmental issues is of high potential to let
decision-makers have the knowledge and means to develop more effective
preventive measures. News articles complement numerical models by describing
extreme situations in detail, documenting the cause and consequences, and
discussing city responses and specific advice [52] for handling heat-related
events. This information offers retrospective evaluations of management
strategies that experts can utilize to refine their approach. However, it is
unclear how to systematically integrate the heterogeneous data types (textual
news and numerical models) for consistent and insightful analysis.

The emergence of LLM offers a new opportunity to address the integration. We
explore how LLMs can be utilized to retrieve relevant news, extract structural
information, and summarize knowledge to facilitate actionable strategies. They
have demonstrated abilities from information extraction [12] to question
answering and even document retrieval [44]. Despite these strengths,
integrating environmental news into heat risk management encounters technical
challenges: First, the overwhelming volume of available news articles
challenges the token limit of LLMs, affecting the efficiency of retrieval;
Second, the task of leveraging LLMs to derive meaningful insights from a large
corpus of news on heat risks remains largely unexplored; Third, the difficulty
of fusing information from varied sources, such as numerical data and textual
news, poses a barrier to obtaining a holistic view of heat risks, further
complicated by the heterogeneous nature of these data sources.

To address those challenges, we implement (Heat Savior), an LLM-empowered
visual analytics system that integrates numerical data and textual news for
heat risk management. provides a novel “thermoglyph” design that utilizes
metaphorical representations to enhance experts’ comprehension of
meteorological conditions. $\system$ aims to support experts in efficiently
retrieving, managing, and navigating a large volume of news articles within a
human-in-the-loop retrieval process through hex bin visualizations of topics
and news glyphs. To distill the news’ semantic meaning, $\system$ employs LLM,
including prompt engineering and retrieval augmented generation (RAG) [37], to
extract structural information and semantic understanding from the news
articles. By combining the strengths of numerical models and the rich contexts
from news articles, $\system$ empowers stakeholders to make informed decisions
and take proactive measures to mitigate the impacts of heat risks. This paper
has the following contributions:

  * $\diamond$

A detailed analysis of heat risk management, identifying key goals and
requirements for design considerations.

  * $\diamond$

A novel LLM-empowered pipeline, including the incorporation of RAG, that
supports human-in-the-loop retrieval and heterogeneous data alignment in the
context of heat risk management.

  * $\diamond$

A VA system, , features “thermoglyph”, news glyphs, and visualization of hex
bins, allowing experts to explore and visualize heat risk insights
interactively. We implement $\system$ through an open-source prototype
system111https://anonymous.4open.science/r/Havior-C58D.

  * $\diamond$

Two case studies and expert interviews to demonstrate the effectiveness of
$\system$, showing that useful insights can be obtained by integrating
numerical and textual data.

##  1 Related Work

###  1.1 Visualizing Climate Risk

Climate risk is commonly defined as the “effect of uncertainty on objectives”
[31], quantified by the probability and impact of hazards such as landslides,
flooding [57, 54], and wildfires [7]. Meteorological visualization research
primarily focuses on displaying and analyzing weather simulation data [51, 43,
4, 30] to derive insights for mitigating these hazards. However, existing
tools often lack the necessary granularity to address the spatial-temporal
complexity of meteorological predictions with adequate explanations [8].
Several visual analytics systems [10, 38, 60, 47] have thereby emerged to
explore the multifaceted spatial-temporal dynamics between meteorological
variables.

However, the analysis of anthropogenic factors, i.e., human-induced causes of
climate change, remains under-explored [43, 45]. An illustrative example is
the dependency on air conditioning during heatwaves. While essential for
coping with rising temperatures, it in turn contributes to climate change and
triggers surges in energy consumption, stressing power grids and amplifying
the risk of grid failures. Existing simulations largely ignore these
anthropogenic effects, indicating a significant research gap in managing
climate risks. The approach we proposed integrates meteorological simulation
results with risk-related news data to provide a novel perspective on
anthropogenic factors, using news as a lens to magnify and segment the
unstructured simulation data stream [29]. This fusion of quantitative
simulations with contextual news analysis offers a logical interpretation of
quantitative variables and empirical support for subjective narratives [28].
Our integrative visualization designs aim to navigate users through both
quantitative and qualitative meteorological phenomena, enhancing data
communication and decision-making in climate risk management. Specifically,
the “thermoglyph” enhances users’ understanding of local climate conditions.
The hex bin layouts for hierarchical tags and news glyphs provide a stratified
analysis of heat risks with progressing details.

###  1.2 Mining Large Text Corpus

Over the past decade, text visualization has focused on microblog data,
notably Tweets, recognizing its utility in providing real-time insights for
disaster response. This data offers a direct glimpse of public sentiment
evolution (e.g., panic during disasters) through topic modeling techniques
like Latent Dirichlet Allocation (LDA), facilitating rapid reactions to
emergencies [57] and public expressions [65]. However, microblogs’ casual
format and unstructured nature present significant challenges, often omitting
necessary context for analyzing periodic events [6]. It revealed the tendency
of such data to amplify biases, whether from frequent posters or through
politicized viewpoints, thus limiting its scope for analyzing broader
meteorological phenomena.

News articles are more reliable in monitoring natural disasters than
microblogs [45]. They often highlight extreme situations, which is a desired
property in analyzing heat risks, and provide in-depth descriptions of local
conditions, consequences, and underlying reasons behind heat-related
incidents. Authored by professionals, these articles are richer, more
coherent, and less noisy in analytical discussions [52] (e.g., expert opinions
and suggested mitigation strategies) than microblogs to support retrospective
evaluations of management strategies. However, the inherent complexity in
these documents requires heterogeneous data sources and advanced techniques
for effective data structuring and interpretation [68, 67, 66]. Moreover,
mapping news to specific climate events has traditionally relied on fuzzy
logic [70] due to the differing detail levels between meteorological data and
journalistic reporting.

To connect news with climate events, textual descriptions can align with
geospatial visualizations for spatial context and information relevance.
Direct references and visual cues within the text can enhance user engagement
and understanding of geospatial context [34, 25, 35, 36]. Our approach draws
inspiration from these design guidelines to maintain close interactions
between text and geospatial visualizations. extracts structured information
from news by LLMs and highlights the key information with novel visual
designs. These designs maintain a good balance between the geospatial context
of numerical models and the semantic understanding of textual information.

###  1.3 Steering Documents Retrieval

One of the great challenges in text mining is to retrieve content of interest
from a vast volume of document corpus [40]. Due to the ambiguity in natural
languages, the retrieval intent can usually be satisfied by multiple similar
filtering conditions. To steer the process of document retrieval, previous
visual analytic systems have proposed solutions based on ranking [6],
similarity [11], and removing overlaps [18]. However, they can easily ignore
some topics of low frequency, which is not negligible in dealing with heat
risk management. As news articles contain dense information, we employ a
hexagon layout to maintain a tight arrangement while retaining text similarity
relations with the neighborhood to further facilitate content-focused
navigation.

With the retrieval results, further computational support should be provided
to facilitate the goal of text mining. Recent advances in LLMs have
demonstrated promise in summarizing large documents, as their pre-trained
intelligence is suitable for a wide range of analytical tasks. Nevertheless,
adapting LLMs to specific domain tasks requires massive computing resources.
Efficient alternatives such as prompt engineering [15] are limited by token
lengths, such that analyzing multiple documents at once remains a challenge.
On the contrary, incorporating a question-based document retrieval system,
DocFlow [50] allows for the intuitive searching of documents by posing
questions in natural language, thereby enhancing both accessibility and search
efficiency. Inspired by these approaches, we explore using RAG, a promising
solution that incorporates knowledge from external databases. This approach is
particularly effective in mitigating common challenges LLMs face, including
creating fabricated content (hallucinations), reliance on outdated
information, and the lack of transparent, traceable reasoning processes [16].
By profoundly integrating LLM for semantics analysis and retrieval of news,
our approach enhances machine reasoning and human-in-the-loop visual analysis,
fostering effective human-AI collaboration in addressing environmental
threats.

##  2 Design Study

Over the past year, our collaboration with two domain experts, E1 and E2,
through weekly meetings, has largely shaped the design of our system. E1 is a
professor in environmental science with more than thirty years of research
experience, while E2 is a specialist in the environmental domain with over
four years of research experience.

Together, we have identified the limitations of numeric domain models. While
these models offer quantitative analysis, they often fail to capture the full
complexity of urbanization, demographics, and socioeconomic factors essential
for developing effective heat risk management strategies. We have collectively
recognized the potential of news sources to bridge this gap. News provides a
factual and direct post-event perspective, enabling a better understanding of
the intricate interplay between these socio-demographic factors and the
numerical data on heat risks. The central challenge we faced was integrating
heterogeneous data sources, including various data types, to create a
comprehensive and human-steerable decision-support pipeline. Through our year-
long collaboration, we have distilled three overarching design goals and six
specific design requirements guiding the development of .

###  2.1 Design Goals

G1: Early warning function for potential heat risks. Given the increasing
impact of climate change, the system should incorporate an early warning
function that alerts experts to potential heat risks based on real-time data
and historical trends, aiming to minimize losses. E2 expressed, “The system is
only considered valuable if it can provide assistance in real-world
scenarios.”

G2: Location-adaptive heat risk analysis. Considering the diverse
meteorological conditions of different regions, employing weather measurements
is insufficient to support bespoke risk management strategies for specific
locations. E1 provides an example: The temperature 28∘ is observed at the
98.6th percentile in Beijing, indicating its rarity. However, the same
temperature is only at the 75.4th percentile in Hong Kong, implying that the
city has more endurance and preparedness for a higher temperature. Despite
experiencing the same temperature in magnitude, the two cities have different
temperature distributions, thus they have different preparedness and
strategies. The magnitude-based metrics (temperature in this case) denote the
meteorological condition at a time point, while probabilistic-based metrics
(percentile) concern multiple measurements in a time period (location-
adaptive). The two metrics should both be considered in heat risk analysis.

G3: Leverage historical knowledge from news for risk comprehension and
informed decision-making. The system should empower experts to understand heat
risks comprehensively and map its insights to the numeric domain model’s
results. The insights include causes, potential consequences, actionable
strategies for decision-makers, and appropriate guidance for citizens. E1
warned, “The 2003 European heatwave, a case often mentioned, resulted in
numerous casualties due to a lack of preparedness and improper instructions.
In the future, we will experience similar extreme summers, or even worse.”

###  2.2 Design Requirements

We defined six requirements that are grouped into three categories: numeric
understanding from climate data analysis, semantic understanding from news,
and integration of them.

R1: Numeric Analyze historical and future trends. To achieve G1, the system
should support historical analysis and future forecasting functions. Experts
should be able to load and analyze numerical data of interest at different
time points, allowing them to examine past trends and forecast future
scenarios effectively.

R2: Numeric Examine spatial meteorological conditions. To reach G2, the system
should enable the analysis of spatial patterns using familiar visualization
forms and analytical methods for domain experts. Features such as spatial zoom
in/out and the ability to switch between different variables should be
provided to facilitate insights.

R3: Semantic Support human-in-the-loop news retrieval. To support G3, a multi-
step retrieval approach should be developed to retrieve a suitable number of
news that align with experts’ interests. E1 and E2 stated, “There are tons of
news for me. I need a system to retrieve an appropriate amount of news in
different stages of analysis.”

R4: Semantic Enhance management and navigation among large-scale news. To
further support G3, with the retrieved news list, efficient management should
allow experts to filter and rank news based on numeric and semantic criteria.
An easy navigation way among a large amount of news and similar news should be
facilitated.

R5: Semantic Extract insights from heat-related news. To address G3, the
system should possess the abilities of structural information extracting,
semantic understanding, and contextual question-answering to help experts gain
semantic insights from heat-related news.

R6: Integration Integrate numeric and semantic insights for decision-making.
For G1, G2, and G3, the system should integrate the figurative impact of news
into the figures in the numeric model’s results to generate location-specific
insights. E1 explained, “If you ask our numeric model how many people die, it
has no answer since it only calculates some meteorological variables.”

##  3 Havior

Figure 1:  The LLM-empowered pipeline contains two parts: data preprocessing
(A) and interactive risk understanding integration (B). The data preprocessing
involves extracting structural information using LLM (A1) and calculating
climate indices (A2-4). In interactive risk understanding integration (B),
heterogeneous understandings are integrated through keywords retrieving (Ba),
topic modeling (Bb), and RAG (Bc). The interactive analysis process is
supported by six views of $\system$ (B1-6) which fulfill the design
requirements.

In this paper, we develop a novel pipeline (Fig. 1) for $\system$ leveraging
the burgeoning capabilities of LLM to integrate numerical data analysis and
semantic understanding to enhance heat risk management.

###  3.1 Data Preprocessing

Climate data were obtained from the ERA5 reanalysis dataset [22]. Its
utilization and performance in heat risk research have been widely
acknowledged for its quality, long-term availability, and accessibility [39,
55]. The hourly data is from 2015 to 2023 and has a spatial resolution of
approximately $27.75\ km\times 27.75\ km$. While it might seem intuitive to
focus on daily temperature extremes (e.g., max and min) to assess heat risks,
studies have shown that daily mean temperature (hereinafter temperature) is
more appropriate because it is more comprehensive in representing the day’s
temperature exposure [61, 20]. The average of 24-hour (in hours UTC)
temperatures within a day was utilized for the temperature for each location.
Moreover, we adopt the Pangu-Weather [3] model with ERA5 temperature estimates
to obtain temperature forecasts up to 14 days (Fig. 1-A3).

To contextualize magnitude-based metrics, we also incorporated probability-
based indicators for a holistic analysis of heat risks.

  * •

Temperature percentiles offer contexts into the local climatology of a
specific region, enabling cross-regional and temporal comparisons [33]. This
is particularly important when considering variations in local tolerance and
preparedness levels [2].

  * •

Return period is a statistical measure indicating the estimated average
interval between the occurrence of heat events. This measure suggests the
likelihood of a certain temperature threshold being exceeded at least once a
year. It evaluates the frequency and intensity of heat events [32].

The heat risk model [17] examines the relationship between heat-related
mortality and temperature fluctuations in 384 locations (Save It for the “Hot”
Day: An LLM-Empowered Visual Analytics System for Heat Risk Management-C1).
The resulting patterns are often represented as “U” curves, with the
temperature range associated with the lowest mortality called the Minimum
Mortality Temperature (MMT). Deviations from the MMT are generally associated
with an exponential increase in the relative risk. Since the local climate
conditions (e.g., tropical or temperate) have a strong influence on individual
locations’ MMT, our analysis adopts the corresponding heat risk models for
each city (Fig. 1-A4).

Environmental news dataset was obtained from Wisers [59], which consisted of
7.7 million environmental (not limited to heat) news articles from Chinese
news publishers, mainly covering East Asian regions. The dataset spans from
July 2015 to June 2023. Each news article contains the title, content,
character statistics, publishing date, publisher, and media type. The media
types encompass both web and publication resources while excluding internet-
based media sources primarily reliant on aggregating news reports from
official news agencies. We extracted structural information from news articles
by using LLM (GPT3.5 [5]), including extracted information (e.g., location,
time, risk description, and consequence) and inferred information (e.g.,
advice and tag) (Fig. 1-A1). This structural information was developed
together with our domain experts (E1-2). The prompts and examples can be found
in the supplementary material.

###  3.2 Interactive Risk Understanding Integration

####  3.2.1 Numerical Climate Data Understanding

Experts can analyze the numerical climate data to gain a quantitative
understanding of the risk (R1, R2). After the numerical data was processed by
the method mentioned before and displayed in numeric panel (Save It for the
“Hot” Day: An LLM-Empowered Visual Analytics System for Heat Risk
Management-A), experts can analyze both magnitude-based and probability-based
indices for heat risk and identify heat hazards. Additionally, the temporal
and spatial information of heat events can help interpret and understand
meteorology. The extracted time and location from news sources are visualized
in the temporal view and spatial view, respectively. Integrating this
information with the climate data facilitates experts’ numerical understanding
of heat risk, which was challenging in the experts’ original workflow. The
interpretability and contextuality it introduced for the result of the climate
numerical model empower experts to gain deeper insights into meteorological
conditions.

By examining the temporal trend (Fig. 1-B1), experts can identify noteworthy
patterns and fluctuations in the meteorological variables and the occurrence
of news events over time (R1). This analysis facilitates the detection of
temporal correlations and the identification of significant events or trends
within the meteorological context. Similarly, the spatial distribution of news
events (Fig. 1-B2) provides valuable information regarding the geographical
patterns and localized impacts of meteorological phenomena (R2). By
visualizing the spatial distribution of news articles, experts can discern
clusters, hotspots, or areas of interest, aiding in the identification of
regions affected by specific weather conditions or events. Furthermore, we
extract numerical temperature from the news. Then we choose news with the
highest number of casualties across different temperatures (Save It for the
“Hot” Day: An LLM-Empowered Visual Analytics System for Heat Risk
Management-C1), allowing us to provide semantic explanations for numerical
risk levels. It transforms the risk level from a mere numerical value into a
relatable example, enabling experts to develop a more intuitive grasp of heat
risks.

####  3.2.2 Topic Understanding & Context Analysis

Experts can utilize the dataset of environment-related news to gain a semantic
understanding of the risk in the city’s context.

Keywords retrieving (Fig. 1-Ba). The first step is to retrieve highly relevant
news using keywords (R3), such as “Hong Kong,” “prolonged,” and “high
temperature,” and automatically filter the retrieved news using semantic
meaning. These keywords can be automatically generated based on the numerical
analysis results or suggested by experts who have analyzed the numerical data.
For instance, a 97.5th percentile with $\geq 4$ days duration [21] can lead to
the inclusion of the keyword “heatwave.” The experts’ choice of cities can
result in the inclusion of the keyword “city name.” To provide experts with
the most relevant news, we use the “is heat risk” and “location” from
structural information to automatically filter the news. In addition to the
automated semantic filtering, we offer experts numeric filters based on the
criteria of “time,” “temperature,” and “casualty,” which can be extracted from
the news.

Topic modeling (Fig. 1-Bb). We harness the capabilities of LLMs to cluster
tags (one item of structural information, Fig. 1-A1) to generate descriptive
topics (R3, R4, R5). They are the concise summarization and categorization of
the retrieved news, facilitating efficient analysis and management (Fig. 1-B3,
Fig. 1-B4). Therefore, it serves two objectives: topic discovery and
retrieval. Utilizing LLM, a wide range of comprehensive topics related to heat
risk in the specific city can be identified. Experts can gain an overview of
the heat risk landscape and uncover unexpected heat risk topics by exploring
these identified topics. Since the number of news articles retrieved about a
city often exceeds the experts’ capacity to effectively read and analyze,
leveraging topics as a criterion enables them to filter the news effectively.
Consequently, they can focus their attention on delving deep into specific
topics.

RAG (Fig. 1-Bc). To continue enhancing the ability to delve into specific
topics, we employ RAG to provide the ranking function based on semantic
meaning (R4). Therefore, experts can use natural language (sentences or
documents) to rank news (Fig. 1-B5). This is achieved by designating the
retrieved news as the knowledge source and leveraging the semantic meaning to
rank the news articles within this source through Embedchain [56]. This
approach allows news that aligns closely with the experts’ interests to be
prioritized and placed higher in the ranked list. Moreover, experts can pose
contextual questions and get more accurate answers from LLM (Fig. 1-B6) by
utilizing the retrieved news articles and the numeric results as the knowledge
source (R5). This integration of news articles and numeric data broadens the
scope of potential questions posed by experts.

####  3.2.3 Heat Risk Management

$\system$ integrates two essential perspectives of analysis: numerical and
semantic (R6) for heat risk management. On the one hand, the numerical
analysis provides valuable insights into the quantitative aspects of the risk,
offering a solid foundation for understanding the magnitude, trends, and
patterns of heat risk. Its results are also utilized for subsequent tasks such
as news retrieval, filtering, and comprehension. On the other hand, the
textual analysis retrieves, filters, clusters, ranks, and analyzes relevant
news articles for an in-depth semantic understanding. This enables experts to
delve deeply into the context, impacts, and complexities surrounding the risk.
Moreover, these insights can assist experts in comprehending and explaining
numerical meteorology. To summarize, the integration of numerical analysis and
semantic understanding allows for a more nuanced assessment, enabling experts
to identify potential correlations, causal relationships, and
interdependencies among various risk factors.

$\system$ provides a summary functionality (Fig. 1-B6) that assists experts in
consolidating their insights, encompassing both numerical and semantic
aspects. These insights, along with the numerical results, are analyzed using
LLM. The analysis culminates in the generation of a comprehensive final
report, which encompasses various elements such as meteorological conditions,
descriptions of the heat risk scenario, historical events or disasters that
have occurred, and corresponding advice for government entities to mitigate
impacts or for citizens to address the risk. This final report serves as a
valuable resource for facilitating informed and well-founded decision-making
by experts, triggering more effective and rational risk management strategies.
Decision-makers can develop proactive risk mitigation plans, allocate
resources effectively, and implement targeted interventions to minimize the
potential negative impacts on communities, economies, and the environment.

##  4 Visual Design of Havior

The interface of $\system$ is shown in the Save It for the “Hot” Day: An LLM-
Empowered Visual Analytics System for Heat Risk Management. To exemplify the
connection between views in the interface, let us consider an expert (Zoe)
utilizing $\system$ for heat risk research. Initially, to check the numerical
meteorological condition, Zoe selects the index, city, and temporal resolution
from the top menus of the meteorological panel. The temporal (R1) and spatial
(R2) climate information is then displayed in the temporal view and spatial
view, respectively. After gaining the numerical understanding, the next step
involves exploring the semantic aspect. Zoe selects the recommended keywords
or types the customized keywords (R3) in the top menus of the news panel. The
news topic view displays topics of the retrieved news using hex bins. Zoe can
make positive or negative selections of hex bins (R3, R4), resulting in
different scatter plots in news glyph view and news lists in the news list.
Zoe can easily locate news of interest with the assistance of news glyph view
and delve into the structural information or full-text of news within the news
list view (R4). The human-in-the-loop retrieval process and the contextual in
the news panel question-answering interface in the summary panel can help her
understand heat-related news (R5). Moreover, insights or knowledge that
experts wish to summarize for subsequent reviewing or report generation (R6)
for decision-making can be pinned to the summary panel. With the integration
of numerical results, $\system$ is able to generate an informative report for
decision-making.

###  4.1 Temporal View

The temporal view (Save It for the “Hot” Day: An LLM-Empowered Visual
Analytics System for Heat Risk Management-A1) is for experts to understand the
temporal trends and distribution of meteorological variables (R1). Considering
the history period (2015-2023) is typically much longer than the future
forecasting period (14 days), line charts for historical and future data are
on the same y-axis, but they are separated on the x-axis. This separation
helps experts distinguish between the known historical data and the projected
future data. To zoom in on a particular timeframe, experts can drag the node
of start or end ().

Furthermore, they need to explore the temporal relationship between trends and
the volume of news related to these parameters (R6). We use the bar chart
(Save It for the “Hot” Day: An LLM-Empowered Visual Analytics System for Heat
Risk Management-A1) to show the number of news articles published. To save
space and easily comparison between the relative magnitudes of changes in both
meteorological data and news volume over time, we combine the bar chart and
the line chart with the dual y-axis with both y-axes beginning from zero. This
design helps in maintaining a visual consistency that can aid in understanding
the relationship between the two datasets (Fig. 4-A, B) without overstating or
understating the variations in either due to scaling issues [26]. In addition
to the temporal trend, we provide a histogram (Save It for the “Hot” Day: An
LLM-Empowered Visual Analytics System for Heat Risk Management-A2) to display
the frequency distribution of temperature (R1).

###  4.2 Spatial View

We provide the spatial view aiming to help experts visualize and comprehend
the spatial distribution of meteorological variables (R2). To achieve this, we
combine the citywide heat map of the variable and the geography map to display
the spatial distribution. In determining the color scheme, we draw inspiration
from ERA5’s color scheme [14], which is chosen based on its association with
human perception of temperature. The spatial relationship between the
distribution of meteorological variables and the geographic locations of news
articles is vital (R6). Thus, we plot news on the map (Save It for the “Hot”
Day: An LLM-Empowered Visual Analytics System for Heat Risk Management-A3).
They will be automatically aggregated in cases of close proximity when zooming
in/out, allowing for a more concise visualization.

Figure 2: The “thermoglyph” in the city gallery for selecting cities. They
employ a metaphorical representation. The pattern of color blocks vividly
depicts the relationship between temperature and percentile for each city. The
black lines connect the current temperature (dashed) or the hovered
temperature (bold) to its corresponding percentile.

Thermoglyph. G2 emphasizes both magnitude-based temperature and probability-
based index. To effectively visualize them, we have developed the
“thermoglyph.” They are presented in the city gallery (Fig. 2), alongside each
city on the map (Save It for the “Hot” Day: An LLM-Empowered Visual Analytics
System for Heat Risk Management-A4), and in the summary panel, which caters to
the varying needs of experts throughout different stages of analysis. The
“thermoglyph” resembles a thermometer, where the color gradually fills from
the bottom to the top, representing the rising mercury in a traditional
thermometer due to heat. The “thermoglyph” consists of two parallel axes:
temperature and percentile. Different temperature ranges and the associated
percentiles are linked and encoded using the same color scheme in the spatial
view. Consequently, unique patterns emerge in different cities (Fig. 2). For
instance, the “thermoglyph” for Hong Kong exhibits a concentrated pattern on
the left side, indicating that the temperature in Hong Kong is more
concentrated for the majority of the time. On the other hand, the
“thermoglyph” for Beijing showcases parallel distribution, representing the
broader temperature range and distinct seasonal characteristics observed in
Beijing. A dashed back line indicates the current value. We also add a solid
black line to accurately illustrate the link when the mouse hovers. This
feature fulfills the experts’ need for precise information.

Design alternatives. We consider a design alternative line chart (Fig. 1-A2)
but find two issues: (1) It is challenging to differentiate between various
patterns based on the changing slopes of the lines and (2) caused confusion
among users because it implies a temporal change. Therefore, we opted for the
design of the “thermoglyph,” which intuitively conveys the relationships
between temperatures and percentiles.

###  4.3 News Topic View

Topic generation of news can help efficiently filter and manage news (R3, R4)
and analysis of news (R5). For news topic view (Save It for the “Hot” Day: An
LLM-Empowered Visual Analytics System for Heat Risk Management-B1), we employ
hex bins with text placed at their centers to represent topics for three
reasons. Firstly, compared to a table list of topics, it has the advantage of
conveying the information of overview. Secondly, the two-dimensional space of
the hex bins preserves the relative spatial relationships between topics [42],
enabling the keeping of semantic meaning relationships. Thirdly, in addition
to being used as a container for displaying topics, hex bins are inherently
well-suited to serve as buttons for experts to filter news. We encode the
quantity of related news for each topic using a grayscale intensity scheme to
avoid confusion with the color in the spatial view.

As mentioned in Section 3.1, we generated tags for each news. Then we employed
the LLM to cluster those tags and generate the title for each cluster,
resulting in a hierarchical structure. The first-level topics, which
correspond to the cluster titles, provide an overview of the information in
each cluster. The second-level topics, which are tags of news, offer more
specific information pertaining to each cluster. The visual design for both
first-level and second-level remains consistent.

For interaction, experts can simply click on first-level topic to display its
second-level topics. By directly double-clicking on a topic, experts can
choose to show or hide the relevant news in the spatial view, news glyph view
and news list, enabling them to focus on specific aspects and in-depth
analysis. Furthermore, when the mouse hovers over a topic, the associated bar
(Save It for the “Hot” Day: An LLM-Empowered Visual Analytics System for Heat
Risk Management-A1) in temporal view and news glyph (Save It for the “Hot”
Day: An LLM-Empowered Visual Analytics System for Heat Risk Management-B2) in
news glyph view will dynamically change color to red.

###  4.4 News Glyph View

Figure 3:  We opt for the coxcomb glyph design (A) for the news glyph.
Alternative design: target glyph design (B) and pie glyph design (C).

The news glyph view (Save It for the “Hot” Day: An LLM-Empowered Visual
Analytics System for Heat Risk Management-B2) aims to leverage experts’
spatial memory for locating and navigating news effectively and enable experts
to identify the news of interest quickly (R4) with each news represented as a
glyph in a 2D space. In our design, we assume that news articles reporting
higher casualties hold greater importance for experts (supported by E1-2).
Thus, we encode the number of deaths, injuries and impacted individuals into
the glyph. By incorporating this information visually, experts can quickly
assess the severity and impact of each piece of news. For locating and
navigating, they require a reasonable layout of news. To achieve this, each
news is first encoded as a high-dimensional vector based on its semantic
meaning and subsequently reduced to two dimensions using UMAP [42],
maintaining semantic similarity. We also used a grid-based method [23] to
avoid clutters and keep the relative distance of news glyphs. When the topic
selection changes, the corresponding news will be shown or hidden without
changing the position to keep the consistency of spatial memory.

We opt for dimension reduction and glyph visualization as opposed to ranking
visualization methods like lineup [19], based on several considerations.
Firstly, news articles encompass a diverse range of topics, and the absence of
casualty information does not necessarily diminish their significance (E2). By
organizing news articles based on their semantic meaning, we ensure that even
those without casualty information are not overlooked entirely. Secondly,
scalability was taken into account. Our glyph design, coupled with a 2D space,
provides an effective solution for accommodating a large number of news
articles. Lastly, the utilization of spatial memory aids in navigation. The
spatial representation allows for better cognitive mapping and recall of
specific articles. Thus, under the condition of a large number of news, we
prioritized an intuitive and space-efficient design. As a result, we opted for
a circular form of glyph to represent each news item.

Coxcomb glyph. For circular glyph form, we designed a modified version of the
classical coxcomb visualization (Fig. 3-A) to represent the numbers of deaths,
injuries, and impacts. To address cases where casualty information is absent,
we added a grey node in the center. This modification ensures that each news
article can be displayed, even if the casualty information is unavailable. The
coxcomb glyph consists of three 120∘ sectors, each distinguished by a
different color to represent deaths, injuries, and impacts. The number of
casualties within each category is encoded by the length of the sector. By
utilizing the coxcomb glyph, experts can easily identify important news that
entails severe consequences. The news glyph is interconnected with both the
news list and spatial view. When a glyph is clicked, the corresponding news
will be automatically centered in the other two views.

Alternative designs. In addition to the coxcomb glyph, we have developed two
alternative designs: the target glyph (Fig. 3-B) and the pie glyph (Fig. 3-C).
However, there are certain limitations associated with them. The target glyph
represents the number of deaths, injuries, and impacts using three individual
arcs. It effectively conveys the information. However, a significant
limitation is that it is spatially expensive. Regardless of whether a news
article contains casualty information or not, the glyph occupies the same
large size. The pie glyph employs size to encode the sum of deaths, injuries,
and impacts. The angle ratio of the sectors is determined based on their
respective numerical ratios. However, the pie glyph has been criticized for
its lack of intuitiveness in expressing individual numbers within each
category. Experts need to consider both size and angle to comprehend the
magnitude of the numbers. Considering these factors, we have chosen the
coxcomb glyph as the preferred design option.

###  4.5 News List View

The news list (Save It for the “Hot” Day: An LLM-Empowered Visual Analytics
System for Heat Risk Management-B3) lists the headlines of news articles. It
provides the functionality for filtering them based on criteria such as time,
temperature (which is closely linked to results of domain models), and
casualties (R3), ranking the news based on their semantic meaning (R4), and
reading original text or structural information (R5). We plot three bar charts
to display the number of news corresponding to each criterion. Then experts
can directly brush the bar to apply the filters. The three bars are
synchronized. When filtering is applied, a blue color emerges in all bar
charts to indicate the number of remaining news.

Experts can use sentences to search and rank news so that they can easily
access the news with similar semantic meanings. To get the details, they can
expand the headlines to structural information or full-text. Additionally,
visual cues are used to highlight relevant sentences pertaining to “risk,”
“cause,” “consequence,” and “advice” within full text, which improves the
efficiency of reading original text (Save It for the “Hot” Day: An LLM-
Empowered Visual Analytics System for Heat Risk Management-B3).

###  4.6 Summary Panel

The summary panel was designed to integrate insights of both numeric and
semantic to facilitate further decisions to minimize the losses of potential
heat risk (R1, R2, R5, R6). To achieve this integration, we incorporate
insights from one side to the other.

Numeric and semantic for numeric. To provide numerical understanding, the line
chart (Save It for the “Hot” Day: An LLM-Empowered Visual Analytics System for
Heat Risk Management-C1) illustrates relationships between city risks and
temperatures. Experts can read the numerical risk level for decision-making.
We also select news with the highest number of deaths at each temperature as
representative examples. This approach allows experts to match impacts with
different risk levels, contextualizing a mere number. These representative
news articles are plotted as scatter points on the x-axis, which show a
tooltip when hovered.

Semantic and numeric for semantic. Any semantic insights found during the
entire process can be pinned and compiled to this panel for the purpose of
reviewing findings and generating the final report. In order to enhance the
comprehension of risk, we implement a contextual question-answering interface
(Save It for the “Hot” Day: An LLM-Empowered Visual Analytics System for Heat
Risk Management-C2) that helps experts pose contextual questions. Contextual
answers will be generated based on selected news articles from the news list
using RAG. The key results derived from the numerical model are rephrased in
natural language for experts to review. They are of utmost importance as they
serve as references for generating the final report as well. By diligently
curating and maintaining the content within this panel, a comprehensive final
report can be generated using LLM. The report becomes a valuable reference for
experts working on future actionable plans.

##  5 Evaluation

For evaluation, besides E1-2 who we closely collaborate with, we involved
three more experts (E3-5) to conduct case study. The panel of experts includes
four professors with an average of 15 years of research experience (33, 10, 9,
and 10 years respectively) and one specialist with 4 years research
experience. Additionally, E1 is a representative in the ‘My Climate Risk’
scheme, launched by the World Climate Research Programme (WCRP) [49] to
mitigate climate event risks.

The experts studied the 2022 extreme heatwave in China [71]. Prior to the case
studies, a training session was conducted to familiarize the experts with the
visual designs, interactions, and connections between different views of
$\system$ [63]. Subsequently, the experts explored $\system$ to analyze their
areas of interest. During the case studies, the experts derived valuable
insights from $\system$. Then we conducted 30-minute interviews with experts
to collect their comments and feedback.

###  5.1 Case Study I - 2022 China Heatwave: Hong Kong

Hong Kong is an exemplary case to delve into the complexities of heat risks in
light of the increasing occurrence of extreme heat events [24].

##### Analysis of Numerical Climate Data

Firstly, experts choose the index, city and temporal resolution as
“temperature,” “Hong Kong” and “daily” to check the meteorological condition.
To research the 2022 China heatwave, experts set the date (single click on
Save It for the “Hot” Day: An LLM-Empowered Visual Analytics System for Heat
Risk Management-A1) to be “2022-07-24” since it has the highest temperature
(Fig. 4-A). The experts commenced their exploration by analyzing the temporal
view (R1) and spatial view (R2), which effectively presented relevant
meteorological information (E2-5). E2 commented, “The geographical
representations, color coding, and glyph design provide an intuitive and
comprehensive means of understanding of the meteorological condition.” They
discovered that the temperature was anticipated to persist at an alarmingly
high level of around 31 degrees Celsius, indicating a prolonged period of
extreme heat (Fig. 4-A). Furthermore, the entire Greater Bay area was engulfed
in high temperatures (Fig. 4-C). Then they checked the temporal distribution
of temperature and found that 26-29 degrees is normal in Hong Kong. So they
don’t know if it is severe. Thus, the experts realized they needed a deeper
understanding of the meteorological condition.

Figure 4:  The temporal trend (A) and the spatial distribution (C) of the
temperature on 2022-07-24 in Hong Kong. The temporal trend (B) and the spatial
distribution (D) of the return period on 2022-07-24 in Hong Kong. Bars in (A)
and (B) are the number of news. Visualizations of the probability-based return
period are better suited for studying extreme heat risk, as compared to
magnitude-based temperature. The consistency of heterogeneous results
(especially in (B)) enhances the interpretability of the system for heat risk
research.

Therefore, they decided to investigate probability-based indices. Using the
“thermoglyph” (Save It for the “Hot” Day: An LLM-Empowered Visual Analytics
System for Heat Risk Management-A4), they gained a holistic understanding of
the relationship between temperature and percentile data (E2-4). To their
surprise, the current percentile for the temperature linked close to the 100th
percentile, indicating an unusually severe situation. Seeking further
validation, they delved into the detailed information on the return periods.
The large return period (Fig. 4-B, D) signifies a low probability of the
temperature occurrence, which raises concerns about the potentially severe
consequences. The analysis of return periods provides additional support to
the conclusion that Hong Kong is confronted with a substantial heat risk (E1,
E3, E5).

##### Analysis of Textual News Data

Experts turned to news to enhance their understanding. They conducted a search
using the recommended keywords, resulting in the retrieval of 2,246 news
articles. By incorporating meteorological variables and news articles (R6),
the researchers discovered that the return period yielded more intriguing
insights (Fig. 4). The consistency observed between the outcomes derived from
heterogeneous climate and news data enhances the rationale for utilizing news
with climate data for heat risk research (E1, E4-5).

Then the experts obtained an overview of the potential heat risks in Hong Kong
by examining topics (R5). The topics of news are automatically generated by
LLM and visualized in the news topic view (Save It for the “Hot” Day: An LLM-
Empowered Visual Analytics System for Heat Risk Management-B1). These heat
risks encompassed various aspects, including well-known topics such as “high
temperature hazard response,” and “impact of climate crisis.” It was not
surprising that the majority of news articles were related to these aspects.
However, the experts also encountered unexpected topics, such as the “water
crisis,” which received relatively less attention. They found this feature to
be highly beneficial in expanding their awareness of previously unrecognized
risk topics. They believed that it would enhance their considerations during
decision-making processes, leading to more informed and rational decisions
(E1, E2, E5). E5 commented, “The topics found here are not preprogramming,
which I think is crucial. When some parameters change or new sources are
included, the result can be automatically updated. This is intrinsically
natural to pick up new things.”

The experts observed a clear seasonal pattern by analyzing the bar plot (Save
It for the “Hot” Day: An LLM-Empowered Visual Analytics System for Heat Risk
Management-A1) showcasing the number of news (R6). Notably, the number of news
articles during the summers of 2018 and 2022 stood out significantly compared
to other years. This observation aligned with the 2018 southern China heatwave
[9] and the 2022 China heatwave [71]. Consequently, experts (E4, E5) could
infer that a severe heatwave would likely occur if $\system$ were used in
2022.

Motivated by these findings, the experts aimed to explore how Hong Kong
responded to heat risks and identify city-based features by deeply delving
into specific topics (R5). This knowledge would enable the development of more
appropriate strategies to assist in preparedness and provide guidance for
governments and citizens alike (E1-5). They decided to explore the most well-
known topics first by filtering news articles (R3) under the categories of
“impact of climate crisis,” excluding others (Save It for the “Hot” Day: An
LLM-Empowered Visual Analytics System for Heat Risk Management-B1). Then they
utilized the news glyph to identify the largest one (Fig. 5-A1) and clicked on
it to access the details (R4). This particular news discussed the Marathon
held in high temperatures that resulted in numerous athlete injuries. While
severe, this outcome was somewhat expected. The experts bookmarked the news
article for future review and pinned important sentences (R4) to the summary
panel, which would contribute to generating the final report.

Subsequently, they came across another news glyph that indicated the highest
number of deaths (Fig. 5-A2). With the help of visual cues for reading the
full text (Fig. 5-B), they can easily gain insights (R5). E3 commented, “The
visualization that maps structural information to the original news text is
remarkably neat and useful. You must have dedicated considerable thought to
this design.”. They found that this news article highlighted the correlation
between high temperatures and increased mortality among mental health
patients, particularly in Hong Kong for its lack of sky view and green space.
The experts found this insight unexpected, as it would have been challenging
to identify this correlation solely based on numerical models.

Figure 5: News glyphs under the topic of “impact of climate crisis” (A). The
largest news glyph (A1) and the news glyph with the largest number of deaths
(A2) are likely to be selected. The structural information highlighted (B) in
the original text helps experts understand the news quickly. The structural
information (C) of the only news with glyph under the topic of “impact of
climate crisis” helps experts understand the news easily. The advice generated
based on summer’s news is similar to what the government has taken [46] in
2018 autumn.

The experts proceeded to investigate the unexpected topic, “water crisis”
(R5). They identified only one news article that has casualty data with the
help of the news glyph. They ranked the list of news based on the semantic
meaning of the selected news (Save It for the “Hot” Day: An LLM-Empowered
Visual Analytics System for Heat Risk Management-B3). Structural information
(Fig. 5-C) is very helpful for quickly analyzing news (E2-4). By examining the
structural information of the first few news in the news list, the experts
quickly discovered that heatwaves can give rise to water supply challenges,
even in a prominent city like Hong Kong. By referring to the red proportion
depicted in the bar (Save It for the “Hot” Day: An LLM-Empowered Visual
Analytics System for Heat Risk Management-A1), they observed that all news
articles concerning tap water supply problems were reported solely during the
previous heatwave in summer of 2018 (R6). Noting the government is criticized
due to the absence of the tap water supply system for remote villages, the
experts posed the question for advice and a considerable answer derived based
on the 2018 summer news selected (Save It for the “Hot” Day: An LLM-Empowered
Visual Analytics System for Heat Risk Management-C2). Based on the observation
that news regarding the unavailability of tap water was only reported during
the summer of 2018, the experts inferred that the government had taken
measures to tackle the issue. Our subsequent investigation proved this
assumption, as evidenced by a government document [46]. Additionally, the
measures taken by the government in 2018 autumn are very similar to the advice
given by $\system$ (Save It for the “Hot” Day: An LLM-Empowered Visual
Analytics System for Heat Risk Management-C2) and structural information (Fig.
5-C). It verifies the ability of $\system$ to discover risk topics and
facilitate informed decision-making (E1-5).

On the contrary, crop risks due to heat risk are inadequately handled. Experts
uncovered another unexpected topic centered around “crop loss” and “crop
damage,” revealing incidents of crop death and economic loss resulting from
unforeseen insect infestations (butterflies that grow in tropical areas have
migrated to Hong Kong) triggered by high temperatures and drought (R5). These
risks were witnessed in 2018, 2021, and recurred in 2022. The lack of
attention and proactive preventive measures towards these issues may explain
their persistence. However, specific measures are expected to have a positive
impact on the risk. we sought the $\system$’s advice and received valuable
advice such as “strengthening pest control,” “improving the irrigation
system,” and “using shade nets to reduce plant heat stress.” By leveraging
$\system$, we not only identify unresolved heat-related risks but also devise
informative strategies to effectively address them (E1-5). E1 commented, “This
is the best practice I want to get from your system. It is helpful for me to
do the risk management.”

##### Summary and Integration of heterogeneous insights (R6)

The experts compiled the semantic insights they discovered in the summary
panel. Alongside these insights, the numerical conclusions were presented.
Additionally, the experts examined the representative news related to the
current ($31^{\circ}$) temperature and identified the need for heightened
attention to the risks faced by outdoor workers. By combining the numerical
conclusions with the semantic insights from the news, the experts generated a
final report on the heat risk in Hong Kong. This report provided them with a
comprehensive understanding of the heat risk and helped them to make informed
decisions and take appropriate actions (E1-2, 4-5). It can be found in the
supplementary material. E2 pointed out, “The convergence of all heterogeneous
data and model generates many insights to me. It is efficient to use LLM to
integrate and summarize them. The report is a valuable reference for informing
subsequent strategy considerations.”

###  5.2 Case Study II - 2022 China Heatwave: Shanghai

Figure 6: The news topic view of Shanghai displays the hierarchical topics of
the retrieved news articles. Left: the first-level topics. Right: the
corresponding second-level topics upon clicking a first-level topic.
Furthermore, double-clicking on a specific topic enables the filtering of news
articles related to that topic in the subsequent analysis process.

To validate the ability of $\system$ to find city-specific insights (R6), we
implemented $\system$ in another city: Shanghai. We examined the heat risk in
Shanghai during the 2022 China heatwave, focusing on identifying any
contrasting city-specific topics compared to Hong Kong. Following a process
similar to that of the first case study, experts examined the meteorological
conditions (R1, R2). Subsequently, they retrieved relevant news using the
keywords “Shanghai” and “high temperature.” Through the analysis of the first-
level topics (Fig. 6), a distinct topic of heat risk: “power crisis” is found
by experts.

They got in-depth knowledge about this topic by learning more subtopics
through the second-level hex bins (R5). After their analysis of the news, they
discovered that the 2022 extreme heatwave, characterized by historically high
extreme temperatures, minimal rainfall, and soaring electricity demand,
exacerbated the power supply-demand imbalance in Shanghai. To mitigate the
impact on economic development and the well-being of residents, Shanghai
implemented measures such as the suspension of landscape lighting to conserve
power. Seeking guidance on this specific issue in Shanghai, the experts sought
assistance from $\system$. They received a comprehensive response, even
including detailed information about the 24-hour power service hotline (95598)
in Shanghai. The contrasting risks faced by Shanghai and the specific advice
generated by the system highlight the significance of considering city-
oriented factors and their unique circumstances in managing heat risks (E1-5).
E4 pointed out, “The system demonstrates its ability to generate insights for
different locations. It is important since the location-oriented strategy for
risk management helps more.”

##  6 Discussion

###  6.1 Findings Discovered and Lessons Learned

Anthropogenic factors in decision-making. In the past, the inclusion of
anthropogenic factors in climate analysis and heat risk management was
hindered by the challenge of modeling these unstructured and non-numeric
elements, which stand in stark contrast to the domain’s usual numerical
models. However, our LLM-empowered pipeline marks a breakthrough, allowing the
seamless incorporation of these factors and their contextualization with the
climate model results. E5 expressed, “By analyzing highly relevant news, the
risk is no longer an abstract number to me. These multifaceted insights help
me make informed decisions.” Our findings reveal that the extensive
documentation (e.g., in news and official documents) of anthropogenic factors
is now accessible for various critical applications, enhanced by the
efficiency of LLMs and the effectiveness of visual analytics.

Yesterday’s news informs tomorrow’s risks. The utility of incorporating
historical news for dealing with unprecedented extreme events has sparked
discussions among both our team and domain experts. A consensus emerged that
while historical insights may have limitations when applied to new or
intensely amplified risk scenarios, they can still aid in identifying risks
with greater magnitude and tracking unresolved risks. E5 believed that
“there’s nothing new under the sun,” suggesting that new heat extremes still
echo those risks in the news. He expressed that identifying existing risks
helps “extend my scope of consideration by imagining how they scale.” The
LLM’s risk summaries enable experts to anticipate and develop preemptive
strategies for more severe extremes. The recurring risks in the news indicate
unresolved problems that need more attention, as in the first case study. On
the other hand, E1 praised the system’s capability to draw connections between
seemingly irrelevant events and reveal new compounded risks. This analysis,
spanning various spatial and temporal dimensions, alerts experts to emerging
risks and suggests proactive measures. E1 noted that the significant changes
brought by several climate events highlight the potential for risks to
converge and intensify, like a chain reaction, eventually creating more
significant issues. He further elaborated, “These risks have been hidden and
cannot be foreseen, but with your system, we now know that they exist and how
they evolve.

Visualizations as the link for LLM and expert expertise. “Why use VA instead
of directly asking LLM for the best heat risk management strategies?” The
answer lies in the fact that domain insights are derived from experts’
exploration. $\system$, as an LLM-empowered VA system, effectively bridges the
gap between LLM and expert expertise by employing carefully designed
visualizations and interactions. E5 thought the LLM is embedded into the
pipeline properly, “Although I think the topics generated by LLM are not
perfect, I find the result combined with hex bins useful to help me discover
new topics. In the original workflow, I would only search for news related to
the topic I had predetermined, which could cause me to miss out on something.”

While new visualizations are beneficial, domain experts may exhibit “inertia.”
We stroke for simple, yet efficient visualizations recommended by experts
during the collaboration. However, experts’ feedback still indicates that the
visual design is helpful but requires time to be adapted. E3 commented, “I
have never seen visualizations like the ‘thermoglyph’ before. However, after a
brief introduction, I understand its efficiency.” The lesson we learned is the
importance of considering the prevalent visualization used in the domain and
the underlying reasons. This awareness enables us to design the most suitable
visualizations that minimize the transition cost.

Applicability and Generalizability We confirmed $\system$’s usefulness and
effectiveness through expert interviews. They specifically appreciated the
vivid “thermoglyph” and the valuable insights they obtained through
interacting with $\system$. In particular, E4 expressed the satisfaction of
effectiveness with $\system$ and complained about the low efficiency and
uninspired results of the manual investigation in their original workflow. E1
expected to use $\system$ in his research and considered the implementation of
$\system$ at the city’s observatory to introduce impactful value. For
generalizability, $\system$ provides a novel and feasible pipeline for
integrating numeric results and insights from textual events. $\system$ can
serve as a source of inspiration for tasks in other domains hurdled by
heterogeneous data, especially numeric and textual. For example, E5 proposed
the potential use of $\system$ in analyzing diverse numeric indices and
financial events in green finance.

###  6.2 Limitations and Future Work

The limitation in LLM, like accuracy, hallucination, domain expertise, and
timeliness, also affects $\system$’s efficiency to provide precise analysis
and up-to-date advice. It is surprisingly difficult to exclude instances like
“real estate market overheating” from heat risks. Despite using the RAG model
to filter timely news articles, we still encounter reliability issues. Our
next step is to refine a domain-specific LLM, aiming for a deeper
comprehension of heat-related risks.

Visual scalability issue arises when the news glyph view displays the topic
with a large volume of news articles. To address this issue, we have designed
a modified coxcomb glyph and implemented a grid-based algorithm to alleviate
the problem of visual clutter. In future research, the exploration of more
advanced visualization techniques and filtering methods holds the potential to
further mitigate this limitation.

Unexplored modalities, such as satellite imagery and video footage about
disasters, also contain information on risk events. We seek to incorporate
multimodality capabilities in $\system$ for risk management.

##  7 Conclusion

In this study, we have undertaken the characterization of the risk management
problem, with a specific focus on the integration challenges posed by the
heterogeneity of numerical results from domain models and risk insights
derived from news sources. We then developed $\system$, an LLM-empowered VA
system, guided by the domain-characterized requirements. $\system$ aims to
enhance the analysis of heat risk, improve heat risk management strategies,
and mitigate heat-related threats. Two case studies and interviews with five
domain experts were conducted for the evaluation. Their positive feedback and
in-depth insights serve as evidence of the usefulness and efficiency of
$\system$. Significantly, the evaluation results demonstrate the potential for
integrating quantitative model results with heterogeneous insights of risk
derived from news reports to enhance the management of heat risks.

## Supplemental Materials

All supplemental materials are available on OSF at
https://osf.io/dx5wt/?view_only=d44d6a39856d4c26b9452fc3f9be64b6, released
under a CC BY 4.0 license. In particular, they include (1) A PDF file
containing LLM Prompts used in the paper and some examples, (2) figure images
in multiple formats, and (3) the process of expert interview.

## Figure Credits

The choropleth about ERA5 in Figure 1A2 is from C3S/ECMWF at
https://climate.copernicus.eu/how-c3s-era5-reanalysis-dataset-can-help-
policymakers. Figure 1Bc is a partial recreation of Fig. 1 from [37], which is
in the public domain.

###### Acknowledgements.

For IEEE VIS, this section may be included in the 2-page allotment for
References, Figure Credits, and Acknowledgments. The authors wish to thank A,
B, and C. This work was supported in part by a grant from XYZ (# 12345-67890).

## References

  * [1] G. Accarino, D. Elia, D. Donno, F. Immorlano, and G. Aloisio.  A machine learning-powered digital twin for extreme weather events analysis.  Technical report, Copernicus Meetings, 2023. doi: 10 . 5194/egusphere-egu23-6060
  * [2] B. G. Anderson and M. L. Bell.  Weather-related mortality: how heat, cold, and heat waves affect mortality in the united states.  Epidemiology, 20(2):205, 2009. doi: 10 . 1097/EDE . 0b013e318190ee08
  * [3] K. Bi, L. Xie, H. Zhang, X. Chen, X. Gu, and Q. Tian.  Accurate medium-range global weather forecasting with 3d neural networks.  Nature, 619(7970):533–538, 2023. doi: 10 . 1038/s41586-023-06185-3
  * [4] A. Biswas, G. Lin, X. Liu, and H.-W. Shen.  Visualization of time-varying weather ensembles across multiple resolutions.  IEEE Transactions on Visualization and Computer Graphics, 23(1):841–850, 2017. doi: 10 . 1109/TVCG . 2016 . 2598869
  * [5] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020. doi: doi . org/10 . 48550/arXiv . 2005 . 14165
  * [6] J. Chae, D. Thom, H. Bosch, Y. Jang, R. Maciejewski, D. S. Ebert, and T. Ertl.  Spatiotemporal social media analytics for abnormal event detection and examination using seasonal-trend decomposition.  In 2012 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 143–152, 2012. doi: 10 . 1109/VAST . 2012 . 6400557
  * [7] C. A. T. Cortes et al.  Analysis of wildfire visualization systems for research and training: Are they up for the challenge of the current state of wildfires?  IEEE Transactions on Visualization and Computer Graphics, pp. 1–20, 2023. doi: 10 . 1109/TVCG . 2023 . 3258440
  * [8] C. V. F. de Souza et al.  Prowis: A visual approach for building, managing, and analyzing weather simulation ensembles at runtime.  IEEE Transactions on Visualization and Computer Graphics, 30(1):738–747, 2024. doi: 10 . 1109/TVCG . 2023 . 3326514
  * [9] K. Deng, S. Yang, D. Gu, A. Lin, and C. Li.  Record-breaking heat wave in southern china and delayed onset of south china sea summer monsoon driven by the pacific subtropical high.  Climate dynamics, 54:3751–3764, 2020. doi: 10 . 1007/s00382-020-05203-8
  * [10] Z. Deng et al.  Airvis: Visual analytics of air pollution propagation.  IEEE Transactions on Visualization and Computer Graphics, 26(1):800–810, 2020. doi: 10 . 1109/TVCG . 2019 . 2934670
  * [11] A. G. Dias, E. E. Milios, and M. C. F. de Oliveira.  Trivir: A visualization system to support document retrieval with high recall.  In Proc. DocEng. ACM, New York, 2019. doi: 10 . 1145/3342558 . 3345401
  * [12] A. Dunn et al.  Structured information extraction from complex scientific text with fine-tuned large language models.  arXiv preprint arXiv:2212.05238, 2022. doi: 10 . 48550/arXiv . 2212 . 05238
  * [13] K. L. Ebi et al.  Hot weather and heat extremes: health risks.  The Lancet, 398(10301):698–708, 2021. doi: 10 . 1016/S0140-6736(21)01208-3
  * [14] ECMWF.  2 m temperature and 30 m wind.  https://charts.ecmwf.int/products/medium-2mt-wind30/, 2024.  [Online; Accessed 15-January-2024].
  * [15] Y. Feng, X. Wang, W. Kam-Kwai, S. Wang, Y. Lu, M. Zhu, B. Wang, and W. Chen.  Promptmagician: Interactive prompt engineering for text-to-image creation.  IEEE Transactions on Visualization and Computer Graphics, 30(1):295–305, 2024. doi: 10 . 1109/TVCG . 2023 . 3327168
  * [16] Y. Feng, X. Wang, B. Pan, W. Kam-Kwai, Y. Ren, S. Liu, Z. Yan, Y. Ma, H. Qu, and W. Chen.  Xnli: Explaining and diagnosing nli-based visual data analysis.  IEEE Transactions on Visualization and Computer Graphics, pp. 1–14, 2023. doi: 10 . 1109/TVCG . 2023 . 3240003
  * [17] A. Gasparrini et al.  Mortality risk attributable to high and low ambient temperature: a multicountry observational study.  The lancet, 386(9991):369–375, 2015. doi: 10 . 1016/S0140-6736(14)62114-0
  * [18] E. Gomez-Nieto et al.  Similarity preserving snippet-based visualization of web search results.  IEEE Transactions on Visualization and Computer Graphics, 20(3):457–470, 2014. doi: 10 . 1109/TVCG . 2013 . 242
  * [19] S. Gratzl, A. Lex, N. Gehlenborg, H. Pfister, and M. Streit.  Lineup: Visual analysis of multi-attribute rankings.  IEEE Transactions on Visualization and Computer Graphics, 19(12):2277–2286, 2013. doi: 10 . 1109/TVCG . 2013 . 173
  * [20] Y. Guo.  The role of humidity in associations of high temperature with mortality: A multicountry, multicity study.  Environmental Health Perspectives, 127, 09 2019. doi: 10 . 1289/EHP5430
  * [21] Y. Guo et al.  Heat wave and mortality: A multicountry, multicommunity study.  Environmental Health Perspectives, 125(8):087006, 2017. doi: 10 . 1289/EHP1026
  * [22] H. Hersbach et al.  The era5 global reanalysis.  Quarterly Journal of the Royal Meteorological Society, 146(730):1999–2049, 2020. doi: 10 . 1002/qj . 3803
  * [23] G. M. Hilasaca, W. E. Marcílio-Jr, D. M. Eler, R. M. Martins, and F. V. Paulovich.  A grid-based method for removing overlaps of dimensionality reduction scatterplot layouts.  IEEE Transactions on Visualization and Computer Graphics, pp. 1–14, 2023. doi: 10 . 1109/TVCG . 2023 . 3309941
  * [24] J. Hua, X. Zhang, C. Ren, Y. Shi, and T.-C. Lee.  Spatiotemporal assessment of extreme heat risk for high-density cities: A case study of Hong Kong from 2006 to 2016.  Sustainable Cities and Society, 64:102507, 2021. doi: 10 . 1016/j . scs . 2020 . 102507
  * [25] N. Ingulfsen, S. Schaub-Meyer, M. Gross, and T. Günther.  News globe: Visualization of geolocalized news articles.  IEEE Computer Graphics and Applications, 42(4):40–51, 2022. doi: 10 . 1109/MCG . 2021 . 3127434
  * [26] P. Isenberg, A. Bezerianos, P. Dragicevic, and J.-D. Fekete.  A study on dual-scale data charts.  IEEE Transactions on Visualization and Computer Graphics, 17(12):2469–2478, 2011. doi: 10 . 1109/TVCG . 2011 . 160
  * [27] P. Jyoteeshkumar reddy, S. E. Perkins-Kirkpatrick, and J. J. Sharples.  Intensifying Australian Heatwave Trends and Their Sensitivity to Observational Data.  Earth’s Future, 9(4):e2020EF001924, 2021. doi: 10 . 1029/2020EF001924
  * [28] W. Kam-Kwai, Y. Luo, X. Yue, W. Chen, and H. Qu.  Prismatic: Interactive multi-view cluster analysis of concept stocks.  arXiv preprint arXiv:2304.05011, 2024. doi: 10 . 48550/arXiv . 2402 . 08978
  * [29] W. Kam-Kwai, X. Wang, Y. Wang, J. He, R. Zhang, and H. Qu.  Anchorage: Visual analysis of satisfaction in customer service videos via anchor events.  IEEE Transactions on Visualization and Computer Graphics, pp. 1–13, 2023. doi: 10 . 1109/TVCG . 2023 . 3245609
  * [30] C. P. Kappe, M. Böttinger, and H. Leitte.  Exploring variability within ensembles of decadal climate predictions.  IEEE Transactions on Visualization and Computer Graphics, 25(3):1499–1512, 2019. doi: 10 . 1109/TVCG . 2018 . 2810919
  * [31] K. Keller, C. Helgeson, and V. Srikrishnan.  Climate risk management.  Annual Review of Earth and Planetary Sciences, 49(1):95–116, 2021. doi: 10 . 1146/annurev-earth-080320-055847
  * [32] H. Kunreuther, G. Heal, M. Allen, O. Edenhofer, C. B. Field, and G. Yohe.  Risk management and climate change.  Nature climate change, 3(5):447–450, 2013. doi: 10 . 1038/nclimate1740
  * [33] W. Lass, A. Haas, J. Hinkel, and C. Jaeger.  Avoiding the avoidable: Towards a European heat waves risk governance.  International journal of disaster risk science, 2:1–14, 2011. doi: 10 . 1007/s13753-011-0001-z
  * [34] S. Latif, S. Chen, and F. Beck.  A deeper understanding of visualization-text interplay in geographic data-driven stories.  Computer Graphics Forum, 40(3):311–322, 2021. doi: 10 . 1111/cgf . 14309
  * [35] S. Latif, Z. Zhou, Y. Kim, F. Beck, and N. W. Kim.  Kori: Interactive synthesis of text and charts in data documents.  IEEE Transactions on Visualization and Computer Graphics, 28(1):184–194, 2022. doi: 10 . 1109/TVCG . 2021 . 3114802
  * [36] F. Lei et al.  Geoexplainer: A visual analytics framework for spatial modeling contextualization and report generation.  IEEE Transactions on Visualization and Computer Graphics, 30(1):1391–1401, 2024. doi: 10 . 1109/TVCG . 2023 . 3327359
  * [37] P. Lewis et al.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  In Proc. NeurIPS. Curran Associates Inc., New York, 2020. doi: 10 . 48550/arXiv . 2005 . 11401
  * [38] D. Liu, K. Veeramachaneni, A. Geiger, V. O. K. Li, and H. Qu.  Aqeyes: Visual analytics for anomaly detection and examination of air quality data.  CoRR, abs/2103.12910, 2021.
  * [39] J. Liu et al.  Mortality burden attributable to high and low ambient temperatures in China and its provinces: results from the global burden of disease study 2019.  The Lancet Regional Health–Western Pacific, 24, 2022. doi: 10 . 1016/j . lanwpc . 2022 . 100493
  * [40] S. Liu et al.  Bridging text visualization and mining: A task-driven survey.  IEEE Transactions on Visualization and Computer Graphics, 25(7):2482–2504, 2019. doi: 10 . 1109/TVCG . 2018 . 2834341
  * [41] S. Mathew, B. Zeng, K. K. Zander, and R. K. Singh.  Exploring agricultural development and climate adaptation in northern australia under climatic risks.  The Rangeland Journal, 40(4):353–364, 2018. doi: 10 . 1071/RJ18011
  * [42] L. McInnes, J. Healy, and J. Melville.  Umap: Uniform manifold approximation and projection for dimension reduction.  arXiv preprint arXiv:1802.03426, 2018. doi: 10 . 48550/arXiv . 1802 . 03426
  * [43] T. Metze.  Visualization in environmental policy and planning: A systematic review and research agenda.  Journal of Environmental Policy & Planning, 22(5):745–760, Sept. 2020. doi: 10 . 1080/1523908X . 2020 . 1798751
  * [44] L. Nan, E. Zhang, W. Zou, Y. Zhao, W. Zhou, and A. Cohan.  On evaluating the integration of reasoning and action in LLM agents with database question answering.  arXiv preprint arXiv:2311.09721, 2023. doi: 10 . 48550/arXiv . 2311 . 09721
  * [45] A. Noviello et al.  Guiding environmental messaging by quantifying the effect of extreme weather events on public discourse surrounding anthropogenic climate change.  Weather, Climate, and Society, 15(1):17 – 30, 2023. doi: 10 . 1175/WCAS-D-22-0053 . 1
  * [46] S. D. C. S. (Oct).  Remote rural water supply plan - shatian meilin village.  https://www.districtcouncils.gov.hk/st/doc/2016_2019/sc/committee_meetings_doc/DHC/13889/st_dhc_2018_047_tc.pdf, 2018.  Publication ID: DH 47 /201 8.
  * [47] R. Palaniyappan Velumani, M. Xia, J. Han, C. Wang, A. K. LAU, and H. Qu.  Aqx: Explaining air quality forecast for verifying domain knowledge using feature importance visualization.  In Proc. IUI, p. 720–733. ACM, New York, 2022. doi: 10 . 1145/3490099 . 3511150
  * [48] L. A. Parsons, D. Shindell, M. Tigchelaar, Y. Zhang, and J. T. Spector.  Increased labor losses and decreased adaptation potential in a warmer world.  Nature communications, 12(1):7286, 2021. doi: 10 . 1038/s41467-021-27328-y
  * [49] W. C. R. Programme.  https://www.wcrp-climate.org/my-climate-risk.  https://www.wcrp-climate.org/my-climate-risk, 2021.  [Online; Accessed 09-January-2024].
  * [50] R. Qiu, Y. Tu, Y.-S. Wang, P.-Y. Yen, and H.-W. Shen.  Docflow: A visual analytics system for question-based document retrieval and categorization.  IEEE Transactions on Visualization and Computer Graphics, 30(2):1533–1548, 2024. doi: 10 . 1109/TVCG . 2022 . 3219762
  * [51] M. Rautenhaus et al.  Visualization in meteorology—a survey of techniques and tools for data analysis tasks.  IEEE Transactions on Visualization and Computer Graphics, 24(12):3268–3296, 2018. doi: 10 . 1109/TVCG . 2017 . 2779501
  * [52] M. Roberts.  What are the heat exhaustion and heatstroke symptoms?  https://www.bbc.com/news/health-62120167, 2023.  [Online; Accessed 08-January-2024].
  * [53] N. P. Simpson et al.  A framework for complex climate change risk assessment.  One Earth, 4(4):489–501, 2021. doi: 10 . 1016/j . oneear . 2021 . 03 . 005
  * [54] L. Styve, C. Navarra, J. M. Petersen, T.-S. Neset, and K. Vrotsou.  A visual analytics pipeline for the identification and exploration of extreme weather events from social media data.  Climate, 10(11), 2022. doi: 10 . 3390/cli10110174
  * [55] S. Sun et al.  Increased moist heat stress risk across China under warming climate.  Scientific Reports, 12(1):22548, 2022. doi: 10 . 1038/s41598-022-27162-2
  * [56] D. Y. Taranjeet Singh.  Embedchain: The open source rag framework.  https://github.com/embedchain/embedchain, 2023.  [Online; Accessed 11-November-2023].
  * [57] D. Thom, R. Krüger, and T. Ertl.  Can twitter save lives? a broad-scale study on visual social media analytics for public safety.  IEEE Transactions on Visualization and Computer Graphics, 22(7):1816–1829, 2016. doi: 10 . 1109/TVCG . 2015 . 2511733
  * [58] A. Vicedo-Cabrera et al.  The burden of heat-related mortality attributable to recent human-induced climate change.  Nature Climate Change, 182, 06 2021. doi: 10 . 1038/s41558-021-01058-x
  * [59] Wisers.  Wisers: best media monitoring.  https://login.wisers.net/, 2024.  [Online; Accessed 01-January-2024].
  * [60] Y. Wu, D. Weng, Z. Deng, J. Bao, M. Xu, Z. Wang, Y. Zheng, Z. Ding, and W. Chen.  Towards better detection and analysis of massive spatiotemporal co-occurrence patterns.  IEEE Transactions on Intelligent Transportation Systems, 22(6):3387–3402, 2021. doi: 10 . 1109/TITS . 2020 . 2983226
  * [61] Z. Xu, J. Cheng, W. Hu, and S. Tong.  Heatwave and health events: A systematic evaluation of different temperature indicators, heatwave intensities and durations.  The Science of the total environment, 630:679–689, 02 2018. doi: 10 . 1016/j . scitotenv . 2018 . 02 . 268
  * [62] J. Yang et al.  Projecting heat-related excess mortality under climate change scenarios in China.  Nature communications, 12(1):1039, 2021. doi: 10 . 1038/s41467-021-21305-1
  * [63] L. Yang, C. Xiong, W. Kam-Kwai, A. Wu, and H. Qu.  Explaining with examples lessons learned from crowdsourced introductory description of information visualizations.  IEEE Transactions on Visualization and Computer Graphics, 29(3):1638–1650, 2023. doi: 10 . 1109/TVCG . 2021 . 3128157
  * [64] W. Yi and A. P. Chan.  Effects of temperature on mortality in Hong Kong: a time series analysis.  International journal of biometeorology, 59:927–936, 2015. doi: 10 . 1007/s00484-014-0895-4
  * [65] F. Yuan, M. Li, and R. Liu.  Understanding the evolutions of public responses using social media: Hurricane matthew case study.  International Journal of Disaster Risk Reduction, 51:101798, 2020. doi: 10 . 1016/j . ijdrr . 2020 . 101798
  * [66] W. Zhang, W. Kam-Kwai, Y. Chen, A. Jia, L. Wang, J.-W. Zhang, L. Cheng, and W. Chen.  Scrolltimes: Tracing the provenance of paintings as a window into history.  IEEE Transactions on Visualization and Computer Graphics, 2024.  To appear. doi: 10 . 48550/arXiv . 2306 . 08834
  * [67] W. Zhang, W. Kam-Kwai, X. Wang, Y. Gong, R. Zhu, K. Liu, Z. Yan, S. Tan, H. Qu, S. Chen, and W. Chen.  Cohortva: A visual analytic system for interactive exploration of cohorts based on historical data.  IEEE Transactions on Visualization and Computer Graphics, 29(1):756–766, 2023. doi: 10 . 1109/TVCG . 2022 . 3209483
  * [68] W. Zhang, S. Tan, S. Chen, L. Meng, T. Zhang, R. Zhu, and W. Chen.  Visual reasoning for uncertainty in spatio-temporal events of historical figures.  IEEE Transactions on Visualization and Computer Graphics, 29(6):3009–3023, 2023. doi: 10 . 1109/TVCG . 2022 . 3146508
  * [69] Q. Zhao et al.  Global, regional, and national burden of mortality associated with non-optimal ambient temperatures from 2000 to 2019: a three-stage modelling study.  The Lancet Planetary Health, 5:E415–E425, 07 2021. doi: 10 . 1016/S2542-5196(21)00081-4
  * [70] M. Zhizhin, E. Kihn, V. Lyutsarev, S. Berezin, A. Poyda, D. Mishin, D. Medvedev, and D. Voitsekhovsky.  Environmental scenario search and visualization.  In Proc. GIS, pp. 1–10. ACM, New York, Nov. 2007. doi: 10 . 1145/1341012 . 1341047
  * [71] B. Zhou, S. Hu, J. Peng, D. Li, L. Ma, Z. Zheng, and G. Feng.  The extreme heat wave in China in August 2022 related to extreme northward movement of the eastern center of SAH.  Atmospheric Research, 293:106918, 2023. doi: 10 . 1016/j . atmosres . 2023 . 106918