# ChIRAAG: ChatGPT Informed Rapid and Automated Assertion Generation

Bhabesh Mali*, Karthik Maddala*, Sweeya Reddy*, Vatsal Gupta*, Chandan Karfa*,
Ramesh Karri†
*Indian Institute of Technology Guwahati, India
†New York University, USA
*{m.bhabesh, k.maddala, p.sweeya, g.vatsal<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

System Verilog Assertion (SVA) formulation– a critical yet complex task– is a
pre-requisite in the Formal Property Verification (FPV) process.
Traditionally, SVA formulation involves expert-driven interpretation of
specifications. This is time-consuming and prone to human error. However,
recent advances in Large Language Models (LLM), LLM-informed automatic
assertion generation is gaining interest. We designed a novel LLM-based
pipeline to generate assertions in English Language, Linear Temporal Logic,
and SVA from natural language specifications. We developed a custom LLM-based
on OpenAI GPT4 for our experiments. Furthermore, we developed testbenches to
verify/validate the LLM-generated assertions. Only 43% of LLM-generated raw
assertions had errors, including syntax and logical errors. By iteratively
prompting the LLMs using carefully crafted prompts derived from test case
failures, the pipeline could generate correct SVAs after a maximum of nine
iterations of prompting. Our results show that LLMs can streamline the
assertion generation workflow, reshaping verification workflows.

###### Index Terms:

Assertion Generation, LLM, Formal Verification

##  I Introduction

Large Language Models (LLMs) have evolved to assist humans in solving complex
problems. LLMs create numerous possibilities for transformative impact across
different domains, including education [1, 2], and finance [3, 4]. The
hardware design industry has used LLMs in executing steps of Electronic Design
Automation (EDA) at a faster rate.

Pre-silicon bug detection is a crucial step in designing functionally correct
hardware. Identification of security vulnerabilities and bugs during the pre-
silicon phase is done via Formal Property Verification (FPV). As per the
functional verification study conducted by Wilson Research Group in 2020 [5],
over 50%, of the overall development resources and expenditures in
Application-Specific Integrated Circuit (ASIC) and Field-Programmable Gate
Array (FPGA) systems were allocated to FPV. FPV is a systematic and rigorous
process of using formal methods to verify whether a system adheres to its
specifications. FPV ensures that the system meets specified properties under
all possible scenarios [6].

Assertions Based Verification (ABV) are used to verify the properties in over
50% of the designs [5]. ABV validates the correct implementation of a design
by integrating safety and liveness properties, expressed as assertions,
throughout the development process. Safety property states that any path
(sequence of events or states) violating the property has a finite prefix
(initial segment of the path) such that every extension of the prefix violates
the property. For example, “whenever a signal request is asserted, signal
acknowledgement is asserted within 3 cycles” is a safety property [7]. While a
liveness property states that any finite path (sequence of events or states)
can be extended to a path satisfying the property [8]. For example, ”Every
request is eventually acknowledged” is a liveness property. However, as the
scale of design and verification requirements increases, so does the number of
assertions, which can reach hundreds, making it too difficult to be proven
formally or falsified within the constraints of available resources manually
[9]. LLMs could perform ABV quickly by automatically generating the correct
assertions. This was the motivation for this study by formulating a novel
pipeline that reduces human effort and increases productivity.

The objective of this work is to analyze how LLMs can assist ABV by generating
assertions automatically from the natural language specifications. We seek to
answer two key questions through this work: RQ1) How good are LLMs in
generating correct assertions in English, Linear Temporal Logic (LTL) and
System Verilog Assertion (SVA) from natural language specification? RQ2) How
correct, complete, and consistent are the generated assertions within the
design and verification framework?. Our contributions are fourfold.

  * •

A framework to study the use of LLM in assertion generation for ABV.

  * •

A prompting method to provide relevant information to an LLM to generate
correct assertions.

  * •

An analysis of performance of LLMs in generating correct assertions using
diverse prompts, verification tools.

  * •

Recommendations for LLMs for assertion generation.

##  II Background and related works

###  II-A Overview of LLM

The journey from Statistical Language Models (SLM) [10, 11, 12] to LLMs is
marked by significant strides in natural language understanding and generation
AI [13]. LLMs have become human assistants revolutionizing science including
medicine [14, 15, 16] and materials science [17, 18]. They enable researchers
to analyze vast datasets, design innovative experiments, and generate creative
hypothesis.

LLMs are Transformer Language models where the construction of the weight
parameters involves the training of models using an extensive volume of text
data acquired from diverse sources such as various websites, web-books, and
PDFs. Based on their functionality, LLMs can be divided into two: General
Domain LLMs (gd-LLMs) and Domain-specific LLMs (ds-LLMs) [19]. gd-LLMs are
trained on diverse resources from various domains and sources, making them
applicable to a wide range of queries. Examples of gd-LLMs include Llama 2
[20] and GPT4 [21]. While ds-LLMs focus on a particular realm of interest.
Examples include MedicalGPT [22] and FoodGPT [23]. LLMs are expensive to train
on large datasets, and hence fine-tuning pre-trained LLMs is becoming
important to simplify training. Fine tuning of an LLM is a supervised process
to adjust the model parameters based on the task [24]. Code Llama [25] by Meta
and StarCoder [26] by Hugging Face are example finetuned LLMs.

###  II-B LLM-Driven Automatic Assertion Generation

LLMs have leading-edge capabilities in diverse tasks including source code
generation [27, 28], automated testing [29, 30], and program repair [31, 32].
Automatic assertion generation using natural language specification has gained
popularity [33]. This study showcased the capability of LLM in generating
assertions solely from RTL code, without additional specifications. Despite
initial syntactic and semantic errors, the authors observed that, with proper
guidance, LLMs consistently produce correct and complete assertions. [34] uses
comments about the security assertions to generate assertions. They assessed
the effectiveness of LLMs in accelerating property verification. They
generated 75,600 assertions across diverse conditions and varying levels of
context and found that LLMs can generate correct assertions at an average rate
of 4.53%. Detailed comments and context proved to be particularly effective in
generating correct assertions.

Use of LLMs in security assertion generation is gaining attention in the
hardware domain. They perform formal property verification to identify any
vulnerability before even before developing the Design Under Test (DUT). The
generated assertions at the pre-silicon level can help monitor and mend
security vulnerabilities during the initial phases of design [6].

###  II-C The Three C’s in ABV

ABV uses assertions which are formal statements or conditions that must always
hold in a design for the proper functioning of the logic. A primary objective
of an assertion is to ensure consistency in the context of the designer intent
and the actual realization of the design [8]. ”Consistency” denotes that there
is no conflict or contradiction between the formal properties/specification
that could lead to ambiguous or erroneous results. Correctness refers to the
properties where it assures that the assertions correctly represent the
intended behavior of the design. Completeness states that the assertions
should cover all related conditions and edge cases within the design. The
verification process shouldn’t miss critical aspects of system behavior. For
example, in a simple digital counter,

⬇

assert @(posedge clk_i) (count == previous_count + 1);

represent a correct assertion that ensures the counter increments correctly
with each clock cycle by checking that the current count is equal to the
previous count plus one. While,

⬇

assert @(posedge clk_i) (enable_signal == (start_signal && !stop_signal));

represents a consistent assertion that ensures control signals are handled
consistently. This case checks that the enable signal is true only when the
start signal is true and the stop signal is false, preventing contradictory
combinations of control signals in the digital counter circuit. And,

⬇

assert @(posedge clk_i) (count >= 0 && count <= MAX_COUNT);

represents a completeness assertion that ensures that the counter’s value
remains within the valid range. It checks that the count is always $\geq 0$
and does not exceed maximum count value (MAX_COUNT)

##  III System Design Methodology and Case Study

This section outlines the LLM pipeline used to generate formal properties
using LLMs and to verify these properties for correctness, consistency and
completeness.

###  III-A Generation of Refined Specification

The initial phase involves collecting descriptions and details of designs from
the OpenTitan Repository [35]. The repository has multiple modules, each
representing a design. A design has multiple contexts including description,
block diagram, details of registers, and Verilog Code. Natural language
specifications are describe in each design from a client perspective.

###  III-B Prompting and Training LLMs

We use OpenAI GPT4 LLM to automatically generate assertions from
specification. LLMs have variability in generating distinct outputs for the
same input. For consistency and completeness, we conducted all experiments on
our design within a consistent chat environment.

####  III-B1 Custom LLM

We formulated a custom GPT4 with ChIRAAG 111 ChIRAAG:
https://chat.openai.com/g/g-ZY5n2KYjD-chiraag. ChIRAAG provides the
functionality of GPT-4 along with Code-Interpreter abilities. ChIRAAG was fed
with initial instructions (Check Instruction file at [36]) and supplied with
vital documents [34, 37, 33, 38, 6] to provide the initial cognition ability
and domain knowledge.

Figure 1: Proposed LLM-assisted ChIRAAG assertion generation flow

####  III-B2 Design-Specific Prompts to LLMs

Figure 2: Block Diagram of RV Timer

Specification of the designs in natural language are provided to the custom
LLM, as an initial prompt. The specification involves detailing the key
functions and generic architecture of the designs, establishing an
understanding of the operational parameters. Additionally, we supplied the
block diagram for each design within their corresponding chat sessions,
seeking a thorough understanding of the flow of information within the system.
We performed our initial experiments on an RTL design, RV Timer, from the
OpenTitan project. The block diagram of RV Timer is depicted in Fig. 2.

The LLM is prompted with the input/output schema, providing an overview of
signal naming and interfacing and added to generate the LTL properties.
Subsequently, the LLM is prompted to give the details of each generated LTL
property in English. This is done to understand if the generated LTL
properties are meaningful, i.e, semantically correct. Next in the ”One time
loop” in Fig. 1, the LLM is supplied with Verilog code of the formal
specification, as provided in the OpenTitan Repository [35]. While the
description of the specification offers a brief overview of various signal
names, however, to ensure an exact match with the signal names and other
variables in the SVA, we have provided the Verilog Code as a prompt. This
ensures that the signal names and other variables in the SV assertions are
closely align with those specified in the Verilog code. This step assists in
developing the test-bench and performing verification. The LLM generates the
raw SVAs along with their English descriptions, that are yet to be verified.
Important assertions of RV Timer are:

⬇

Assertion 1:

// Assertion to check if tick_count resets to 0 on reset

property p_reset_tick_count;

@(posedge clk_i) (!rst_ni) |-> (tick_count == 12’h0);

endproperty

assert property (p_reset_tick_count);

Assertion 2:

// Assertion to check if tick_count resets to 0 when ’active’ is deasserted

property p_deactivate_tick_count;

@(posedge clk_i) (!active) |-> (tick_count == 12’h0);

endproperty

assert property (p_deactivate_tick_count);

Assertion 3:

// Assertion to check if tick_count increments correctly

property p_tick_count_increment;

@(posedge clk_i) (active && (tick_count < prescaler)) |-> (tick_count ==
$past(tick_count) + 1);

endproperty

assert property (p_tick_count_increment);

####  III-B3 Training-Specific Prompts

After generating raw SVAs using LLM, we developed a test bench for the
property verification process. We use Synopsys VCS 2021.09 simulator to
validate the SVAs. Verification involves deploying test cases in the testbench
to verify these assertions222EDA Playground allowed for small-scale testing..
Whenever an assertion is modified following a prompt to the LLM, we
systematically re-verify the assertions, checking for ambiguity or errors
using the logs of SVA assertion checker tool. If an error is detected by the
checker, the LLM is instructed, based on the information in the log file of
the assertion checker tool, to recheck, understand, rectify, and generate new
assertions. This process involved a human in the feedback loop to refine
prompts. Some LLM-generated assertions are inconsistent and have syntax
errors. For example,

⬇

Assertion 2:

// Assertion to check if tick_count resets to 0 on reset

property p_reset_tick_count;

@(posedge clk_i) (!rst_ni) |-> (tick_count == 12’h0);

endproperty

assert property (p_reset_tick_count);

The above assertion is inconsistent and incorrect when the log file is
supplied as a prompt to the LLM. The LLM reported:

⬇

Message:

Issue: The assertion is failing because tick_count is not resetting to 0 when
it reaches the prescaler value.

Potential cause: There might be an off-by-one error in the comparison or a
timing issue where the reset of tick_count does not occur in the same cycle as
tick_count equals prescaler.

Fix: Inspect the conditional block in the always_ff block that is supposed to
reset tick_count. Make sure that the condition for resetting (tick_count ==
prescaler) is accurate and that the reset occurs at the correct timing.

Subsequently, the LLM is prompted to fix the errors.

⬇

Prompt: Please give the modified and correct assertions and fix the
inconsistency

In the first training prompt itself, the LLM identified the cause of the error
to be an operator mishandling and provided the correct assertion, as later
verified by the SVA checker tool. The LLM outputted the correct assertion as:

⬇

Assertion 2:

// Assertion to check if tick_count resets to 0 on reset

property p_reset_tick_count;

@(posedge clk_i) (!rst_ni) |=> (tick_count == 12’h0);

endproperty

assert property (p_reset_tick_count);

Some assertions require multiple prompts due to multiple errors such as
logical errors and missing signals. For example,

⬇

Assertion 3:

// Assertion to check if tick_count increments correctly

property p_tick_count_increment;

@(posedge clk_i) (active && (tick_count < prescaler)) |-> (tick_count ==
$past(tick_count) + 1);

endproperty

assert property (p_tick_count_increment);

The assertion is reported incorrect and inconsistent when log file is
supplied. The LLM initially detailed the error as:

⬇

Message:

Issue: This assertion fails because tick_count is not incrementing as expected
when active is high and tick_count is less than prescaler.

Potential cause: This could be due to the tick_count not being properly
incremented in the logic when active is high and tick_count < prescaler, or
there could be timing issues where the increment does not align with the clock
edge as expected.

Fix: Verify the logic in the always_ff block that handles the increment of
tick_count. Ensure that the condition for incrementing tick_count is correctly
implemented and synchronizes with the clock edge.

The LLM is further prompted as:

⬇

Prompt: Please give the modified and correct assertions and fix the
inconsistency

The assertion output by LLM after modification is:

⬇

Assertion 3:

// Assertion to check if tick_count increments correctly

property p_tick_count_increment;

@(posedge clk_i) (active && (tick_count < prescaler) && rst_ni) |->
(tick_count == $past(tick_count) + 1);

endproperty

assert property (p_tick_count_increment);

Surprisingly, the LLM understood the missing signal rst_ni from the learned
information and included it in the generated assertion. However, the assertion
fails after certain time. When the log is again provided as a prompt to the
LLM, it generates the following message:

⬇

Message:

Failure Log: The log indicates a failure at 6 ns with the assertion
’(tick_count == ($past(tick_count) + 1))’.

Observation: The failure occurs just after the reset is deasserted (rst_ni
goes high at 4 ns), suggesting an issue in the handling of the reset
deassertion.

and a revised assertion as:

⬇

Assertion 3:

// Assertion to check if tick_count increments correctly

property p_tick_count_increment;

@(posedge clk_i)

if (rst_ni && $past(rst_ni, 1) && active && (tick_count < prescaler))

|-> (tick_count == $past(tick_count) + 1);

else if (rst_ni && !$past(rst_ni, 1))

|-> (tick_count == 0);

endproperty

assert property (p_tick_count_increment);

The assertion is still incorrect and failed test cases at certain instances
when re-verified with the checker. We prompted the LLM with log files, with
syntax errors as most frequently occurred, up to four times.

⬇

Prompt: The log is again giving this syntax error:

Parsing design file ’design.sv’

Error-[SE] Syntax error

Following verilog source has syntax error :

"design.sv", 77: token is ’else’

else if (rst_ni && !$past(rst_ni, 1))

Please understand the given error and modify the assertion

This log error indicates that assertion has incorrect syntax;

⬇

Message: It seems the issue is with the use of else if within the property
block. In SystemVerilog, sequential logic inside properties often needs to be
structured differently compared to typical procedural code. Let’s revise the
property to correctly handle the conditional logic.

with the following assertion generated:

⬇

Assertion 3:

// Assertion to check if tick_count increments correctly

property p_tick_count_increment;

@(posedge clk_i)

(rst_ni && $past(rst_ni, 1) && active && (tick_count < prescaler)) ->
(tick_count == $past(tick_count) + 1) |

(rst_ni && !$past(rst_ni, 1)) -> (tick_count == 0);

endproperty

assert property (p_tick_count_increment);

This whole process generated two distinct assertions, including the previously
mentioned one with modification:

⬇

Assertion 3a:

// Assertion for tick_count increment

property tick_count_increment;

@(posedge clk_i)

(rst_ni && active && (tick_count < prescaler)) -> (tick_count ==
$past(tick_count) + 1);

endproperty

assert property (tick_count_increment);

Assertion 3b:

// Assertion for tick_count reset on reset deassertion

property tick_count_reset_on_reset_deassert;

@(posedge clk_i)

(!$past(rst_ni, 1) && rst_ni) -> (tick_count == 0);

endproperty

assert property (tick_count_reset_on_reset_deassert);

One of these assertions is inconsistent and fails test cases at certain
timestamps. The LLM reported it as:

⬇

Message:

Assertion Failure: (((rst_ni && active) && (tick_count < prescaler)) ->
(tick_count == ($past(tick_count) + 1)))

Observation: The failure suggests that either the tick_count does not
increment as expected when rst_ni and active are high and tick_count is less
than prescaler, or there is an issue with the timing of when tick_count
increments.

and produced the correct assertion as:

⬇

Assertion 3a:

// Assertion for tick_count increment

property tick_count_increment;

@(posedge clk_i)

(rst_ni && active && (tick_count < prescaler)) |=> (tick_count ==
$past(tick_count) + 1);

endproperty

assert property (tick_count_increment);

Similar errors are observed in other assertions. Prompting plus logs generated
by the SVA checker can re-generate correct and consistent assertions to pass
simulation runs. Simulation reports, test benches and assertions are in [36].

Figure 3: Synopsys VCS 2021.09 FPV Report

##  IV Discussion and Conclusion

We observed that the initial raw assertion, generated by ChIRAAG based on
specification and knowledge of the block diagram, might contain flaws that are
reported by the assertion checker. The most common ones are syntax errors and
manual intervention prompted by a domain-expert can address them. LLMs produce
additional relevant information fulfilling the meaningfulness of the generated
assertions. We can answer the questions in the Introduction. Indeed, an LLM is
a good starting point to assist an engineer in assertion generation. Only 43%
of ChIRAAG-generated assertions needed modifications to pass all the test
cases. Almost all these assertions had some form of syntactic errors, while a
few had inconsistencies. We presented ChIRAAG with real errors as they
appeared in the log, rather than using predefined natural language sentences.
Surprisingly, the LLM successfully identified most of the errors within a
single prompt and rectified them to pass the test cases. While LLMs can
identify syntax, semantic, and inconsistency failures, it can also recognize
missing or overlooked signals and include them in the generated assertions
later on during prompting process.

The signal rst_ni was overlooked in the initial raw assertion, as required in
Assertion 3. However, LLM was able to perceive and include it. Similarly, LLM
was able to identify logical errors and generate the extra assertions later,
passing all test cases. We observed that LLMs were providing information about
the issues the assertion has, the potential cause for the error, and
information about fixing the errors. This helps root-cause the failure and
create appropriate prompts. Although the assertions are complete in terms of
satisfying the intended behavior captured by the test bench, it is not
possible to confirm 100% functional coverage of design intent. Fig. 3 shows
logs after successful simulation, satisfying all test cases.

TABLE I: Performance Report OpenTitan RTL | Assertions | # Iterations
---|---|---
| OpenTitan | LLM | to Converge
RV Timer | 0 | 9 | 9
PattGen | 0 | 7 | 5
GPIO | 0 | 6 | 10
HMAC | 4 | 12 | 15
ROM Controller | 6 | 14 | 16
sram_controller | 0 | 11 | 3
adc_controller | 5 | 7 | 5

We performed similar experiments for seven designs from the OpenTitan Project
and produced correct and consistent assertions. Table I is the performance
report LLM-based assertion generation. For each design, the LLM generated
correct and consistent assertions after some number of iterative prompting. “#
of iterations to converge ” is the number of iterations required to generate
the correct and consistent assertions, i.e., test cases are satisfied. The
third column shows the number of assertions generated by LLM for each of the
designs, the second column titled “OpenTitan Assertions” shows the assertions
provided with the RTL. LLM generated these “OpenTitan Assertions”
successfully, despite initial variation in variable names. Interestingly, the
LLM generates more assertions than was provided for the design in OpenTitan.
For example, LLM generates 12 assertions for HMAC while 4 assertions are given
for the design. Importantly, LLM generated assertions are actually important
and cover some aspects of the design which was are missed by the OpenTitan
assertions. The assertion:

⬇

assert @(posedge clk) disable iff (rst_n)

(msg_fifo_req && hmac_en) |-> ##1 reg_hash_start)

else $fatal("Assertion failed");

is crucial to check the HMAC process start property. The HMAC process should
start whenever there is a request in the message FIFO and HMAC signal is
enabled. That is, if ‘msg_fifo_req’ and ‘hmac_en’ are true, then the signal
‘req_hash_start’ must be true on the next rising edge of the clock, unless a
reset (‘rst_n’) is active. If this condition is violated, a fatal error
message is triggered. While in the ROM Controller, the additional SVA below,
generated by LLM besides the provided OpenTitan assertion is important:

⬇

Assertion 1:

assert @(posedge clk) (checker_done && (q !== ’Checking’ && q !== ’Done’)) |->
##1 (q === ’Invalid’);

Assertion 2:

assert @(posedge clk) (counter_done |-> (q !== ’Reading Low’ && q_next !==
’Invalid’)

Assertion 3:

assert @(posedge clk) (((q !== ’Invalid’) && rst_ni && kmac_rom_last_o) |->
##1 kmac_rom_vld_o)

These assertions verify important properties: Assertion 1 checks if
’checker_done’ is true when the system state (’q’) is not in the ’Checking’ or
’Done’ states. If the condition is violated, the assertion requires that in
the next clock cycle, the state ’q’ must be set to invalid. Assertion 2 states
that if the counter_done signal is true, then the state ’q’ is either not
”Reading Low’ in the current state or does not become ’Invalid’ in the next
state. While, Assertion 3 states that if the current state ’q’ is not
’Invalid’ and both the reset condition ’rst_ni’ and the signal
’kmac_rom_last_o’ are true, then in the next clock cycle, the signal
’kmac_rom_vld_o’ should be true.

This shows usefulness of the LLM in generating assertions for ABV of designs.
However, to be highlighted, LLM requires some manual checks to ensure
correctness and consistency of the properties, i.e, it needs repeated
prompting to ensure self-correction in some cases. While the LLM is useful, it
may not be flawless in generating assertions from specifications. Providing
the block diagram of the design can enhance understanding of the various
signals and the precise flow. Overall, LLM is useful in orchestrating the
design verification process, showcasing its promise to revolutionize
verification.

This study focused on two C’s, i.e., correct and consistent assertions from
English language specification. We will develop LLM prompts to check the third
C i.e., completeness of assertions with respect to the specification.

## References

  * [1] P. Sharma _et al._ , “Performance of chatgpt on usmle: Unlocking the potential of large language models for ai-assisted medical education,” _arXiv preprint: 2307.00112_ , 2023.
  * [2] S. Elkins _et al._ , “How useful are educational questions generated by large language models?” in _International Conference on Artificial Intelligence in Education_. Springer, 2023\.
  * [3] S. Wu _et al._ , “Bloomberggpt: A large language model for finance,” _arXiv preprint arXiv:2303.17564_ , 2023.
  * [4] X. Zhang _et al._ , “Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters,” in _Proceedings ACM Conf on Information and Knowledge Management_ , 2023.
  * [5] H. Foster, “Part 1: The 2020 wilson research group functional verification study,” Feb 2021. [Online]. Available: https://tinyurl.com/verification-foster-group
  * [6] H. Witharana _et al._ , “Automated generation of security assertions for rtl models,” _ACM Journal on Emerging Technologies in Computing Systems_ , vol. 19, no. 1, pp. 1–27, 2023.
  * [7] A. F. V. T. Committee _et al._ , “Psl property specification language reference manual,” 2004.
  * [8] H. D. Foster _et al._ , _Assertion-based design_. Springer Science & Business Media, 2004.
  * [9] D. Wang _et al._ , “Automatic assume guarantee analysis for assertion-based formal verification,” in _Proc. Asia and South Pacific Design Automation Conference_ , 2005, pp. 561–566.
  * [10] F. Jelinek, _Statistical methods for speech recognition_. MIT press, 1998.
  * [11] R. Rosenfeld, “Two decades of statistical language modeling: Where do we go from here?” _Proceedings of the IEEE_ , 2000.
  * [12] J. Gao _et al._ , “Introduction to the special issue on statistical language modeling,” 2004.
  * [13] W. X. Zhao _et al._ , “A survey of large language models,” _arXiv preprint arXiv:2303.18223_ , 2023.
  * [14] A. J. Thirunavukarasu _et al._ , “Large language models in medicine,” _Nature medicine_ , 2023.
  * [15] S. Gilbert _et al._ , “Large language model ai chatbots require approval as medical devices,” _Nature Medicine_ , 2023.
  * [16] E. Waisberg _et al._ , “Large language model (llm)-driven chatbots for neuro-ophthalmic medical education,” _Eye_ , 2023.
  * [17] K. M. Jablonka _et al._ , “14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon,” _Digital Discovery_ , 2023.
  * [18] A. N. Rubungo _et al._ , “Llm-prop: Predicting physical and electronic properties of crystalline solids from their text descriptions,” _arXiv preprint arXiv:2310.14029_ , 2023.
  * [19] C. Wang _et al._ , “Survey on factuality in large language models: Knowledge, retrieval and domain-specificity,” _arXiv preprint arXiv:2310.07521_ , 2023.
  * [20] H. Touvron _et al._ , “Llama 2: Open foundation and fine-tuned chat models,” _arXiv preprint arXiv:2307.09288_ , 2023.
  * [21] OpenAI, “Gpt-4 technical report,” 2023.
  * [22] M. Xu, “Medicalgpt: Training medical gpt model,” https://github.com/shibing624/MedicalGPT, 2023.
  * [23] Z. Qi _et al._ , “Foodgpt: A large language model in food testing domain with incremental pre-training and knowledge graph prompt,” 2023.
  * [24] Z. Zhang _et al._ , “Balancing specialized and general skills in llms: The impact of modern tuning and data strategy,” 2023.
  * [25] B. Roziere _et al._ , “Code llama: Open foundation models for code,” _arXiv preprint arXiv:2308.12950_ , 2023.
  * [26] R. Li _et al._ , “Starcoder: may the source be with you!” _arXiv preprint arXiv:2305.06161_ , 2023.
  * [27] M. Chen _et al._ , “Evaluating large language models trained on code,” _arXiv preprint arXiv:2107.03374_ , 2021.
  * [28] Y. Li, “Competition-level code generation with alphacode,” _Science_ , vol. 378, no. 6624, pp. 1092–1097, 2022.
  * [29] Y. Deng _et al._ , “Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt,” _arXiv preprint arXiv:2304.02014_ , 2023.
  * [30] C. S. Xia _et al._ , “Universal fuzzing via large language models,” _arXiv preprint arXiv:2308.04748_ , 2023.
  * [31] C. S. Xia, “Less training, more repairing please: revisiting automated program repair via zero-shot learning,” in _Proceedings of ACM Joint European Software Engineering Conference and Symposium on Foundations of Software Engineering_ , 2022.
  * [32] N. Jiang _et al._ , “Impact of code language models on automated program repair,” _arXiv preprint arXiv:2302.05020_ , 2023.
  * [33] M. Orenes-Vera, “Using llms to facilitate formal verification of rtl,” 2023.
  * [34] R. Kande _et al._ , “Llm-assisted generation of hardware assertions,” 2023\.
  * [35] OpenTitan, “OpenTitan Repository,” https://opentitan.org/, Visited: 2023\.
  * [36] B. Mali, “LLM-Generated-SVA,” https://github.com/BhabeshMali/LLM-Generated-SVA, 2023.
  * [37] P. Srikumar, “Fast and wrong: The case for formally specifying hardware with llms,” in _Proc. ACM ASPLOS_ , 2023.
  * [38] P. Dasgupta, _A roadmap for formal property verification_. Springer, 2006.


# The geometry of Riemannian submersions from compact Lie groups. Application
to flag manifolds

Dimbihery Rabenoro and Xavier Pennec
Université Côte d’Azur and Inria Epione team, France

###### Abstract

We study the geometry of Riemannian manifolds of finite dimension which are
isometric to a quotient of a compact Lie group endowed with the quotient of a
bi-invariant metric. Then, we apply the results obtained to enlight the
geometry of flag manifolds.

##  1 Introduction

In this paper, we study the geometry of Riemannian manifolds of finite
dimension which are isometric to a quotient of a compact Lie group endowed
with the quotient of a bi-invariant metric Grove 2000 (Grove, 2000), Kerin and
Shankar 2012 (Kerin and Shankar, 2012). Such a manifold is thus a Riemannian
submersion from the compact Lie group, over which the latter is a principal
bundle Guijarro and Walschap 2007 (Guijarro and Walschap, 2007). The initial
motivation for our work was to enlight the geometry of flag manifolds.

Flags are properly nested sequences of linear subspaces Alekseevsky 1996
(Alekseevsky, 1996), Monk 1959 (Monk, 1959). They can equivalently be
described as sequences of mutually orthogonal incremental subspaces whose
partial sums generates the nested sequence. Flag manifolds have recently been
a rising subject of interest in statistical machine learning for generalizing
Grassmanians and handling multiple subspaces at the same time, as in Nishimori
Akaho and Plumbley 2006 (Nishimori Akaho and Plumbley, 2006). It was also
recently shown that Principal Component Analysis (PCA) can be rephrased as an
optimization on the Flag. One of the novelty of Pennec 2018 (Pennec, 2018) is
to realize that the nested sequence of linear subspaces actually optimizes a
criterion on the flag manifold: the accumulated unexplained variance (AUV). On
this basis, new robust variants like the minimal accumulated unexplained
p-variance have been proposed, even for manifold-valued data Pennec 2017
(Pennec, 2017).

The ubiquitous appearance of flags in other domains such as numerical analysis
exemplified in Ye Wong and Lim 2022 (Ye Wong and Lim, 2022) led these authors
to develop the mathematical tools for Riemannian optimization algorithms on
the flag manifold. This landmark paper describes flag spaces in multiple ways
with different geometric structures. However, it is not always easy to
disentangle what is due to each structure. In this paper, we clarify the
structure of flag manifolds as Riemannian submersions from the orthogonal
group. In that purpose, we develop the general setting described at the
beginning of this introduction and prove that flag manifolds fall within this
framework.

For an arbitrary compact Lie group ${\mathbb{G}}$ endowed with a bi-invariant
metric $g^{{\mathbb{G}}}$, the key point is the decomposition of smooth vector
fields as linear combination of left-invariant vector fields with coefficients
which are smooth functions on ${\mathbb{G}}$. Now, the manifold
$({\mathbb{B}},g^{{\mathbb{B}}})$ which we study is a Riemannian submersion
from $({\mathbb{G}},g^{{\mathbb{G}}})$. Applying O’Neill’s formula O'Neill
1966 (O’Neill, 1966), we derive notably the Levi-Civita connection and the
curvature tensors of $({\mathbb{B}},g^{{\mathbb{B}}})$. In particular, we
obtain closed-forms of these Riemannian objects in the case of flag manifolds.

In Section 2, we describe the general framework and the basic geometry of
$({\mathbb{B}},g^{{\mathbb{B}}})$. Then, in Section 3, we prove that flag
manifolds can be placed inside this setting. Finally, in Section 4, we
establish the formulas for the Levi-Civita connection and the curcurvature
tensors of $({\mathbb{B}},g^{{\mathbb{B}}})$.

##  2 The general framework

######  Theorem 1.

Let ${\mathbb{B}}$ be a subset of a manifold $M$. Let ${\mathbb{G}}$ be a
compact lie group, endowed with a bi-invariant metric $g^{{\mathbb{G}}}$,
which acts smoothly from the left on $M$. For $p\in M$, let ${\mathbb{K}}(p)$
be its isotropy group. Assume that

$(i)$ ${\mathbb{B}}$ is the orbit of some $p_{0}\in M$.

$(ii)$ The action of ${\mathbb{K}}(p_{0})$ on ${\mathbb{G}}$ by right-
multiplication is isometric.

Let $\psi:{\mathbb{G}}\longrightarrow{\mathbb{G}}/{\mathbb{K}}(p_{0})$ be the
canonical map and $\pi$ the map defined by
$\pi:{\mathbb{G}}\longrightarrow{\mathbb{B}}$ and $\pi(Q)=Q\cdot p_{0}$. Then,

$(1)$ ${\mathbb{B}}$ is an embedded submanifold of $M$ and $\pi$ induces a
diffeomorphism
$\overline{\pi}:{\mathbb{G}}/{\mathbb{K}}(p_{0})\simeq{\mathbb{B}}$ such that
$\overline{\pi}\circ\psi=\pi$.

$(2)$ The quotient metric $\overline{g}^{{\mathbb{G}}}$ on
${\mathbb{G}}/{\mathbb{K}}(p_{0})$ is well-defined and $\pi$ is a Riemannian
submersion from $({\mathbb{G}},g^{{\mathbb{G}}})$ onto
$({\mathbb{B}},g^{{\mathbb{B}}})$ ,where
$g^{{\mathbb{B}}}:=\overline{\pi}_{*}\overline{g}^{{\mathbb{G}}}$.

$(3)$ $({\mathbb{G}},\pi,{\mathbb{B}},{\mathbb{K}}(p_{0}))$ is a principal
fiber bundle, i.e. with typical fiber the Lie group ${\mathbb{K}}(p_{0})$.

This is represented by the following diagram.

$\textstyle{{\mathbb{G}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi}$$\scriptstyle{\pi}$$\textstyle{{\mathbb{G}}/{\mathbb{K}}(p_{0})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\overline{\pi}}$$\scriptstyle{\sim}$$\textstyle{{\mathbb{B}}\hookrightarrow
M}$

###### Proof.

$(1)$ is standard: See Proposition A.2. in Bendokat Zimmermann and Absil 2020
(Bendokat Zimmermann and Absil, 2020). For $(2)$, by section 29.21 in Michor
2008 (Michor, 2008), $(ii)$ implies that $\overline{g}^{{\mathbb{G}}}$ is
well-defined and that $\psi$ is a Riemannian submersion from
$({\mathbb{G}},g^{{\mathbb{G}}})$ onto
$({\mathbb{G}}/{\mathbb{K}}(p_{0}),\overline{g}^{{\mathbb{G}}})$. Finally,
$(3)$ is a direct consequence of Lemma 18.3 in Michor 2008 (Michor, 2008).
Indeed, ${\mathbb{K}}(p_{0})$ acts freely on ${\mathbb{G}}$ by right-
multiplication and the orbits of this action are the fibers of $\pi$. ∎

######  Remark 1.

It is not necessary in the proof of Theorem 1 to assume that
$g^{{\mathbb{G}}}$ is bi-invariant. However we use this condition later, so we
suppose it from the beginning.

###  2.1 Vertical and horizontal parts

Denote by $e$ the neutral element of ${\mathbb{G}}$ and, for
${\widetilde{Q}}\in{\mathbb{G}}$, by
${\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}$ the left-translation by
${\widetilde{Q}}$ in ${\mathbb{G}}$: for all $Q\in{\mathbb{G}}$,
${\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}(Q)={\widetilde{Q}}Q$. Let
${\mathfrak{g}}$ be its Lie algebra. Set
$N:=\dim({\mathfrak{g}})=\dim({\mathbb{G}})$. For all
$\Omega,\Omega^{\prime}\in{\mathfrak{g}}$, set

${\llbracket}\Omega,\Omega^{\prime}{\rrbracket}:=g_{e}\left(\Omega,\Omega^{\prime}\right).$

When ${\mathbb{G}}$ is a matrix Lie group, i.e. ${\mathbb{G}}\subset
GL_{n}({\mathbb{R}})$ for $n\geq 1$, ${\mathfrak{g}}=M_{n}({\mathbb{R}})$ and

${\llbracket}\Omega,\Omega^{\prime}{\rrbracket}:=\Omega\Omega^{\prime}-\Omega^{\prime}\Omega.$

For all $Q\in{\mathbb{G}}$, the vertical and horizontal subspaces in
$T_{Q}{\mathbb{G}}$ are respectively $V_{Q}^{\pi}:=\ker(d\pi)_{Q}$ and
$H_{Q}^{\pi}:=\left(V_{Q}^{\pi}\right)^{\perp}$. We also introduce the linear
projections ${\mathrm{ver}}_{Q}^{\pi}:T_{Q}{\mathbb{G}}\longrightarrow
V_{Q}^{\pi}$ and ${\mathrm{hor}}_{Q}^{\pi}:T_{Q}{\mathbb{G}}\longrightarrow
H_{Q}^{\pi}$. The next lemma states that the vertical and horizontal parts are
invariant wrt left-translations in ${\mathbb{G}}$.

######  Lemma 1.

For $Q,{\widetilde{Q}}\in{\mathbb{G}}$ and $\Delta\in T_{Q}{\mathbb{G}}$, set
${\widetilde{Q}}_{*}(\Delta):=\left(d{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}\right)_{Q}\left(\Delta\right)$.
Then, for all $\Omega\in{\mathfrak{g}}$,

${\mathrm{ver}}_{{\widetilde{Q}}Q}^{\pi}\left(\left({\widetilde{Q}}Q\right)_{*}\Omega\right)={\widetilde{Q}}_{*}{\mathrm{ver}}_{Q}^{\pi}\left(Q_{*}\Omega\right)$
(2.1)

and

${\mathrm{hor}}_{{\widetilde{Q}}Q}^{\pi}\left(\left({\widetilde{Q}}Q\right)_{*}\Omega\right)={\widetilde{Q}}_{*}{\mathrm{hor}}_{Q}^{\pi}\left(Q_{*}\Omega\right).$
(2.2)

###### Proof.

${\mathbb{G}}$ acts from the left on ${\mathbb{B}}\subset M$, so that for all
$Q,{\widetilde{Q}}\in{\mathbb{G}}$, ${\widetilde{Q}}\cdot(Q\cdot
p_{0})=({\widetilde{Q}}Q)\cdot p_{0}$, i.e.
${\widetilde{Q}}\cdot\pi(Q)=\pi({\widetilde{Q}}Q)$. This means that
${\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{B}}}\circ\pi=\pi\circ{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}$,
where, for any $p\in{\mathbb{B}}$,
${\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{B}}}(p):={\widetilde{Q}}\cdot p$.
This implies that for any $Q_{*}\Omega\in T_{Q}{\mathbb{G}}$,

$\left(d{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{B}}}\right)_{\pi(Q)}\left(\left(d\pi\right)_{Q}\left(Q_{*}\Omega\right)\right)=\left(d\pi\right)_{{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}(Q)}\left(\left(d{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}\right)_{Q}\left(Q_{*}\Omega\right)\right)=\left(d\pi\right)_{{\widetilde{Q}}Q}\left({\widetilde{Q}}_{*}\left(Q_{*}\Omega\right)\right).$

Since $\left(d{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{B}}}\right)_{\pi(Q)}$
is an isomorphism, we deduce that: $Q_{*}\Omega\in
V_{Q}^{\pi}\iff{\widetilde{Q}}_{*}\left(Q_{*}\Omega\right)\in
V_{{\widetilde{Q}}Q}^{\pi}$. Since $g^{{\mathbb{G}}}$ is a left-invariant
metric, this latter equivalence implies the following one: $Q_{*}\Omega\in
H_{Q}^{\pi}\iff{\widetilde{Q}}_{*}\left(Q_{*}\Omega\right)\in
H_{{\widetilde{Q}}Q}^{\pi}$. Now,

$Q_{*}\Omega={\mathrm{ver}}_{Q}^{\pi}\left(Q_{*}\Omega\right)+{\mathrm{hor}}_{Q}^{\pi}\left(Q_{*}\Omega\right)\implies\left({\widetilde{Q}}Q\right)_{*}\Omega=\left({\widetilde{Q}}\right)_{*}{\mathrm{ver}}_{Q}^{\pi}\left(Q_{*}\Omega\right)+\left({\widetilde{Q}}\right)_{*}{\mathrm{hor}}_{Q}^{\pi}\left(Q_{*}\Omega\right).$
(2.3)

By the preceding equivalences, $(\ref{implyVH})$ yields the decomposition of
$\left({\widetilde{Q}}Q\right)_{*}\Omega$ into its vertical and horizontal
parts in $T_{{\widetilde{Q}}Q}{\mathbb{G}}$, which concludes the proof. ∎

###  2.2 Basic geometry of the base

Recall that for $p\in{\mathbb{B}}$ and $Q\in\pi^{-1}(p)$, the restriction of
$\left(d\pi\right)_{Q}$ to $H_{Q}^{\pi}$ is an isomorphism between
$H_{Q}^{\pi}$ and $T_{p}{\mathbb{B}}$. For $\Delta\in T_{p}{\mathbb{B}}$, the
horizontal lift wrt $\pi$ of $\Delta$ at $Q$ is denoted by
$\Delta^{\sharp}_{Q}$. It is the unique vector in $H_{Q}^{\pi}$ whose image by
$(d\pi)_{Q}$ is $\Delta$. Then, $g^{{\mathbb{B}}}$ is defined by

$g^{{\mathbb{B}}}_{p}(\Delta,\widetilde{\Delta})=g_{Q}^{{\mathbb{G}}}(\Delta^{\sharp}_{Q},\widetilde{\Delta}^{\sharp}_{Q}).$
(2.4)

######  Proposition 1.

$({\mathbb{B}},g^{{\mathbb{B}}})$ is a complete Riemannian manifold and the
geodesic in ${\mathbb{B}}$ through $p$ in the direction $\Delta$ is of the
form

$t\mapsto\pi\left({\mathcal{L}}_{Q}^{{\mathbb{G}}}\left(\exp_{{\mathfrak{g}}}\left(t\left(Q^{-1}\right)_{*}\Delta_{Q}^{\sharp}\right)\right)\right),$

where $Q\in\pi^{-1}(p)$ and
$\exp_{{\mathfrak{g}}}:{\mathfrak{g}}\longrightarrow{\mathbb{G}}$ is the Lie
group exponential map of ${\mathbb{G}}$.

###### Proof.

Since ${\mathbb{G}}$ is compact, ${\mathbb{B}}$ is complete and by the Hopf-
Rinow theorem, $({\mathbb{B}},g^{{\mathbb{B}}})$ is complete. Since
$g^{{\mathbb{G}}}$ is bi-invariant, the geodesics of
$({\mathbb{G}},g^{{\mathbb{G}}})$ are left-translates of one-parameter
subgroups of ${\mathbb{G}}$. Now, by Lemma 1,
$\left(Q^{-1}\right)_{*}\Delta_{Q}^{\sharp}\in H_{e}^{\pi}$. So, the curve
$t\mapsto{\mathcal{L}}_{Q}^{{\mathbb{G}}}\left(\exp_{{\mathfrak{g}}}\left(t\left(Q^{-1}\right)_{*}\Delta_{Q}^{\sharp}\right)\right)$
is the horizontal geodesic in ${\mathbb{G}}$ through $Q$, in the direction
$\Delta_{Q}^{\sharp}$. Thus, its image by $\pi$ is indeed the geodesic in
${\mathbb{B}}$ through $p$ in the direction $\Delta$. ∎

##  3 A new representation of flag manifolds

###  3.1 Some usual representations of flag manifolds

A flag of vector spaces in ${\mathbb{R}}^{n}$ is a filtration $\mathcal{V}$ of
subspaces $V_{0}=\\{0\\}\subset V_{1},\ldots\subset V_{r}={\mathbb{R}}^{n}$.
For all $1\leq i\leq r$, let $W_{i}$ be the orthogonal complement of $V_{i-1}$
in $V_{i}$. Then, $(W_{i})_{1\leq i\leq r}$ is a sequence of mutually
orthogonal linear subspaces of ${\mathbb{R}}^{n}$, called the incremental
subspaces representation of the flag $\mathcal{V}$. The sequence
${\mathrm{I}}=(q_{i})_{1\leq i\leq r}$, where $q_{i}:=\dim(W^{i})$, is called
the type of $\mathcal{V}$. The set of all flags of type ${\mathrm{I}}$ is
denoted by $\Phi^{{\mathrm{I}}}$.

On one hand, $\Phi^{{\mathrm{I}}}$ is represented as follows: see Ye Wong and
Lim 2022 (Ye Wong and Lim, 2022). For $1\leq q\leq n$, we denote by
${G_{n}^{q}}$ the Grassmannian of all linear subspaces of dimension $q$ of
${\mathbb{R}}^{n}$ and we identify ${G_{n}^{q}}$ with orthogonal projectors.
Then, we define the product $G^{{\mathrm{I}}}$ of Grassmanians associated to
the type ${\mathrm{I}}$ by
$G^{{\mathrm{I}}}:=\prod\limits_{i=1}^{r}G_{n}^{q_{i}}$. In this projector
perspective, $\Phi^{{\mathrm{I}}}$ is identified with the submanifold
${\mathcal{F}^{\mathrm{I}}}$ of $G^{{\mathrm{I}}}$ defined by

${\mathcal{F}^{\mathrm{I}}}:=\left\\{\mathcal{P}=(P_{i})_{1\leq i\leq r}\in
G^{\textrm{I}}:\leavevmode\nobreak\
P_{i}^{2}=P_{i}=P_{i}^{T},\leavevmode\nobreak\
P_{i}P_{j}=0,\leavevmode\nobreak\ i\neq j\right\\}.$ (3.1)

On the other hand, in Nguyen 2021 (Nguyen, 2021), the following representation
of $\Phi^{{\mathrm{I}}}$ is considered. Let $(\lambda_{i})_{1\leq i\leq r}$ be
distinct real numbers. Consider the diagonal matrix
$\mathcal{D}_{0}=\mathrm{Diag}\left[\lambda_{1}I_{q_{1}},...,\lambda_{r}I_{q_{r}}\right]$,
where $I_{q_{i}}$ is the identity matrix of order $q_{i}$, and the left-action
of the orthogonal group $O(n)$ on $\mathrm{Sym_{n}}$ defined by $Q\cdot
S:=QSQ^{T}$. Then, $\Phi^{{\mathrm{I}}}$ is identified with
$O(n)\cdot\mathcal{D}_{0}\subset\mathrm{Sym_{n}}$, i.e. the orbit of
$\mathcal{D}_{0}$. Set

$O(\mathrm{I}):=\prod\limits_{i=1}^{r}O(q_{i}).$

Then, through this representation, $\Phi^{{\mathrm{I}}}$ is an embedded
submanifold of $\mathrm{Sym_{n}}$ which is diffeomorphic to
$O(n)/O({\mathrm{I}})$.

A similar formalism is developed in Bendokat Zimmermann and Absil 2020
(Bendokat Zimmermann and Absil, 2020) to study the geometry of ${G_{n}^{q}}$.
Namely, consider the projector
$P_{0}:=\mathrm{Diag}\left[I_{q},0_{n-q}\right]\in\mathrm{Sym_{n}}$. Then,
${G_{n}^{q}}$ is the orbit of $P_{0}$ under the same action of $O(n)$ on
$\mathrm{Sym_{n}}$ and the map $\pi^{q}:O(n)\longrightarrow{G_{n}^{q}}$
defined by $\pi^{q}:Q\longmapsto Q\cdot P_{0}$ is a Riemannian submersion.
Therefore, the geometry of ${G_{n}^{q}}$ is described in terms of that of
$O(n)$.

In this paper, we prove that ${\mathcal{F}^{\mathrm{I}}}$ is the orbit a fixed
flag ${\mathcal{P}}_{0}$ under a suitable action of $O(n)$ on
$G^{{\mathrm{I}}}$. Analogously to the case of ${G_{n}^{q}}$, the combination
of the projector perspective and the Riemannian submersion setting allows to
derive results on the geometry of the flag manifold
${\mathcal{F}^{\mathrm{I}}}$.

###  3.2 Flag manifolds in the general framework

The compact Lie group $O(n)$ is endowed with the bi-invariant metric $g^{O}$
defined by

$g^{O}_{Q}\left(\Delta,\widetilde{\Delta}\right):=\frac{1}{2}\mathrm{tr}\left(\Delta^{T}\widetilde{\Delta}\right),\quad
Q\in{\mathbb{G}},\leavevmode\nobreak\ \Delta,\widetilde{\Delta}\in T_{Q}O(n).$

The group $O(n)$ acts smoothly on $G^{{\mathrm{I}}}$ by
$(Q,\mathcal{P})\mapsto
Q\cdot\mathcal{P}:=\left(QP_{1}Q^{T},...,QP_{r}Q^{T}\right)$. Introduce the
standard flag ${\mathcal{P}}_{0}^{{\mathrm{I}}}:=\left(P_{0}^{i}\right)_{1\leq
i\leq r}\in{\mathcal{F}^{\mathrm{I}}}$, where

$P_{0}^{i}:=\mathrm{Diag}\left[0_{q_{1}},...,I_{q_{i}},...,0_{q_{r}}\right],\quad
1\leq i\leq r.$

Clearly, the isotropy group of ${\mathcal{P}}_{0}^{{\mathrm{I}}}$ is
isomorphic to $O({\mathrm{I}})$. Now, this action stabilizes
${\mathcal{F}^{\mathrm{I}}}$, so that we may consider the map
$\pi^{{\mathrm{I}}}:O(n)\longrightarrow{\mathcal{F}^{\mathrm{I}}}$ defined by

$\pi^{{\mathrm{I}}}\left(Q\right)=Q\cdot\mathcal{P}_{0}^{{\mathrm{I}}}=\left(QP_{i}^{0}Q^{T}\right)_{1\leq
i\leq r}.$

######  Proposition 2.

${\mathcal{F}^{\mathrm{I}}}$ satisfies the conditions of Theorem 1 with
$M=G^{{\mathrm{I}}}$, ${\mathbb{G}}=O(n)$, $g^{{\mathbb{G}}}=g^{O}$,
$p_{0}={\mathcal{P}}_{0}$ and ${\mathbb{K}}(p_{0})=O({\mathrm{I}})$. So, the
map ${\pi^{\mathrm{I}}}$ is a Riemannian submersion and induces a principal
bundle structure over ${\mathcal{F}^{\mathrm{I}}}$.

###### Proof.

We need to prove that $(i)$ and $(ii)$ of Theorem 1 hold. For any
${\mathcal{P}}=(P_{i})_{1\leq i\leq r}\in{\mathcal{F}^{\mathrm{I}}}$, let
$Q\in O(n)$ such that for all $1\leq i\leq r$, its $i$-th block $Q^{(i)}$ of
columns is an orthonormal basis of $\mathrm{Im}(P^{i})$. Then, for all $1\leq
i\leq r$, $Q^{-1}P_{i}Q=P_{0}^{i}$ and
$\left(QP_{0}^{1}Q^{T},...,QP_{0}^{r}Q^{T}\right)={\mathcal{P}}$. This proves
that the orbit of ${\mathcal{P}}_{0}$ is ${\mathcal{F}^{\mathrm{I}}}$, i.e.
$(i)$ holds. Since the metric $g^{O}$ is right-invariant, $(ii)$ holds. ∎

###  3.3 Basic geometry of flag manifolds

####  3.3.1 Tangent spaces

For any $\mathcal{P}=\left(P_{i}\right)_{i}\in\mathcal{F}^{\mathrm{I}}$, the
tangent space $T_{\mathcal{P}}\mathcal{F}^{\mathrm{I}}$ is a linear subspace
of $T_{\mathcal{P}}G^{\textrm{I}}=\bigoplus_{i=1}^{r}T_{P_{i}}G^{i}$. By
differentiating the relation $P_{i}P_{j}=0$ in $(\ref{flagProj})$, we obtain
that

$T_{\mathcal{P}}\mathcal{F}^{\mathrm{I}}=\left\\{\Delta=(\Delta^{i})_{1\leq
i\leq r}\in T_{\mathcal{P}}G^{\mathrm{I}}:\forall\leavevmode\nobreak\ i\neq
j,\leavevmode\nobreak\ \Delta^{i}P^{j}+P^{i}\Delta^{j}=0\right\\}.$ (3.2)

######  Lemma 2.

For all $Q\in O(n)$ and $Q\Omega\in T_{Q}O(d)$ with
$\Omega\in\mathfrak{so}(n)$,

$\left(d{\pi^{\mathrm{I}}}\right)_{Q}\left(Q\Omega\right)=\left(Q\left[\Omega,P_{0}^{i}\right]Q^{T}\right)_{i}.$

###### Proof.

Let $\gamma:[0,1]\longrightarrow O(n)$ be a smooth curve with $\gamma(0)=Q$
and $\gamma^{\prime}(0)=Q\Omega$. Then,

$\left(d{\pi^{\mathrm{I}}}\right)_{Q}\left(Q\Omega\right)=\left.\frac{d}{dt}\right|_{t=0}\pi^{\mathrm{I}}\left(\gamma(t)\right)=\left(\left.\frac{d}{dt}\right|_{t=0}\gamma(t)P_{0}^{i}\gamma(t)^{T}\right)_{i}.$

Now $\Omega^{T}=-\Omega$. So, for all $1\leq i\leq r$,

$\left.\frac{d}{dt}\right|_{t=0}\gamma(t)P_{0}^{i}\gamma(t)^{T}=\left(Q\Omega\right)P_{0}^{i}Q^{T}+QP_{0}^{i}\left(Q\Omega\right)^{T}=Q\left[\Omega,P_{0}^{i}\right]Q^{T}.$

∎

By Lemma 2, for any $Q\in O(n)$,
$V_{Q}^{{\pi^{\mathrm{I}}}}=\left\\{Q\Omega\in
T_{Q}O(d):\left[\Omega,P_{0}^{i}\right]=0,\leavevmode\nobreak\ 1\leq i\leq
r\right\\}$. The computation of $\left[\Omega,P_{0}^{i}\right]$ implies that

$V_{Q}^{{\pi^{\mathrm{I}}}}=\left\\{Q\mathrm{Diag}\left[\Omega_{ii}\right]:\Omega_{ii}\in\mathfrak{so}(q_{i}),\leavevmode\nobreak\
1\leq i\leq r\right\\}.$

Therefore,

$H_{Q}^{{\pi^{\mathrm{I}}}}=\left\\{Q\Omega:\Omega\in\mathfrak{so}(n),\leavevmode\nobreak\
\widetilde{\Omega}_{ii}=0,\leavevmode\nobreak\ 1\leq i\leq r\right\\}.$

In particular, we recover that
$\dim\left({\mathcal{F}^{\mathrm{I}}}\right)=\dim\left(H_{Q}^{{\pi^{\mathrm{I}}}}\right)\sum\limits_{i>j}q_{i}q_{j}$.

####  3.3.2 Horizontal lifts

######  Lemma 3.

For all $\Omega\in{\mathfrak{so}(n)}$,

${\mathrm{hor}}_{e}^{\pi^{{\mathrm{I}}}}(\Omega)=\frac{1}{2}\sum\limits_{i=1}^{r}{\llbracket}{\llbracket}\Omega,P_{0}^{i}{\rrbracket},P_{0}^{i}{\rrbracket}.$

###### Proof.

This is a consequence of computations of block-matrices in
${\mathfrak{so}(n)}$, partitioned wrt the type ${\mathrm{I}}$. ∎

######  Proposition 3.

For all ${\mathcal{P}}=(P_{i})_{i}\in{\mathcal{F}^{\mathrm{I}}}$,
$\Delta=(\Delta_{i})_{i}\in T_{{\mathcal{P}}}{\mathcal{F}^{\mathrm{I}}}$ and
$Q\in\left(\pi^{{\mathrm{I}}}\right)^{-1}({\mathcal{P}})$,

$\Delta^{\sharp}_{Q}=\frac{1}{2}\left(\sum\limits_{i=1}^{r}{\llbracket}\Delta_{i},P_{i}{\rrbracket}\right)Q.$

###### Proof.

This follows from the preceding Lemma and the description of the tangent
spaces to ${\mathcal{F}^{\mathrm{I}}}$. ∎

####  3.3.3 Metric, geodesics and Exponential map

For $\mathcal{P}=\left(P_{i}\right)_{i}\in\mathcal{F}^{\mathrm{I}}$ and
$\Delta,\widetilde{\Delta}\in T_{{\mathcal{P}}}{\mathcal{F}^{\mathrm{I}}}$,

$g^{{\mathcal{F}^{\mathrm{I}}}}_{{\mathcal{P}}}\left(\Delta,\widetilde{\Delta}\right)=\frac{1}{4}\sum\limits_{i,j}g^{O}_{e}\left({\llbracket}\Delta_{i},P_{i}{\rrbracket},{\llbracket}\widetilde{\Delta}_{j},P_{j}{\rrbracket}\right)$

######  Remark 2.

This metric should be compared with that of Ye Wong and Lim 2022 (Ye Wong and
Lim, 2022), which is the restriction of that of $G^{{\mathrm{I}}}$.

######  Proposition 4.

For $\mathcal{P}=\left(P_{i}\right)_{i}\in\mathcal{F}^{\mathrm{I}}$ and
$\Delta=\left(\Delta_{i}\right)_{i}\in
T_{\mathcal{P}}\mathcal{F}^{\mathrm{I}}$, the geodesic of
$\mathcal{F}^{\mathrm{I}}$ starting at $\mathcal{P}$ in the direction $\Delta$
is of the form
$t\longmapsto\pi^{\mathrm{I}}\left(Q\exp_{m}\left(tQ^{T}\overline{\Omega}Q\right)\right)$,
where $\exp_{m}$ is the matrix exponential and

$\overline{\Omega}=\frac{1}{2}\sum\limits_{i=1}^{r}{\llbracket}\Delta_{i},P_{i}{\rrbracket}.$

Then,

$\mathrm{Exp}_{\mathcal{P}}^{\mathcal{F}^{\mathrm{I}}}\left(\Delta\right)=\left(\exp_{m}\left(\overline{\Omega}\right)P_{i}\exp_{m}\left(-\overline{\Omega}\right)\right)_{i}.$
(3.3)

###### Proof.

Since $\overline{\Omega}=Q^{T}\overline{\Omega}Q$ and $QP_{i}^{0}Q^{T}=P_{i}$,
the following computation implies that $(\ref{expMap})$ holds:

$\mathrm{Exp}_{\mathcal{P}}^{\mathcal{F}^{\mathrm{I}}}\left(\Delta\right)=\pi^{\mathrm{I}}\left(Q\exp_{m}\left(Q^{T}\overline{\Omega}Q\right)\right)=\pi^{\mathrm{I}}\left(\left(\exp_{m}\left(\overline{\Omega}\right)\right)Q\right)=\left(\exp_{m}\left(\overline{\Omega}\right)QP_{i}^{0}Q^{T}\exp_{m}\left(\overline{\Omega}\right)\right)_{i}.$

∎

######  Remark 3.

This should be compared with the Exponential map in Ye Wong and Lim 2022 (Ye
Wong and Lim, 2022).

######  Remark 4.

There exist some algorithms for computing the Riemannian Logarithm of
Riemannian submersions.

##  4 Levi-Civita connection and curvature

Throughout the sequel, in order to alleviate the notations, we assume that
${\mathbb{G}}$ is a matrix Lie group, that is ${\mathbb{G}}\subset
GL_{n}({\mathbb{R}})$, for $n\geq 1$. This means that
${\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}$ and
$\left(d{\mathcal{L}}_{{\widetilde{Q}}}^{{\mathbb{G}}}\right)_{Q}$ are both
replaced by left-multiplication by ${\widetilde{Q}}$. The results obtained
under this assumption can easily be generalized to an arbitrary compact Lie
group.

We shall make use, at times, of the following assumption.

######  Assumption 1.

For all $\Omega,\Omega^{\prime}\in H^{\pi}_{e}$,
${\llbracket}\Omega,\Omega^{\prime}{\rrbracket}\in V^{\pi}_{e}$.

###  4.1 Decomposition of vector fields

The idea is to decompose any smooth vector field on ${\mathbb{G}}$ into a
linear combination of left-invariant vector fields with coefficients which are
smooth functions on ${\mathbb{G}}$. This is possible thanks to the following
tool.

######  Proposition 5.

A $1$-form $\omega$ on a manifold $M$ is smooth if and only if for every
smooth vector field $X$ on M, the function $\omega(X)$ is smooth on $M$.

###### Proof.

We omit the proof. This is standard. ∎

######  Corollary 1.

Let $(\epsilon_{k})_{1\leq k\leq N}$ be any basis of ${\mathfrak{g}}$. Let
$\left(E_{k}\right)_{k}$ be the left-invariant vector fields on
${\mathbb{G}}$, defined by $E_{k}(Q)=Q_{*}\epsilon_{k}$, for
$Q\in{\mathbb{G}}$. Then, for any $U\in{\mathcal{X}}\left({\mathbb{G}}\right)$
and $Q\in{\mathbb{G}}$,

$U_{Q}=\sum\limits_{k=1}^{N}u_{k}(Q)E_{k}(Q)=Q_{*}\left(\sum\limits_{k=1}^{N}u_{k}(Q)\epsilon_{k}\right),$

where the $u_{k}$’s are smooth functions on ${\mathbb{G}}$. Let
$\overline{U}:{\mathbb{G}}\longrightarrow{\mathfrak{g}}$ be the map defined by

$\overline{U}(Q)=\left(Q^{-1}\right)_{*}U_{Q}=\sum\limits_{k=1}^{N}u_{k}(Q)\epsilon_{k}.$

Then, $\overline{U}$ is a smooth map. Furthermore,

$\left(d\overline{U}\right)_{Q}=\sum\limits_{k=1}^{N}\left(du_{k}\right)_{Q}\epsilon_{k}.$

###### Proof.

Apply Proposition 5.

∎

###  4.2 Levi-Civita connection

######  Proposition 6.

Let $U,V\in{\mathcal{X}}({\mathbb{G}})$. Then,

$\left(\nabla^{{\mathbb{G}}}_{U}V\right)_{Q}=Q\left(d\overline{V}\right)_{Q}\left(U_{Q}\right)+\frac{Q}{2}{\llbracket}\overline{U}(Q),\overline{V}(Q){\rrbracket}.$
(4.1)

###### Proof.

First, for $1\leq k\leq\ell\leq N$, $E_{k}$ and $E_{\ell}$ are livf’s. So, for
any $Q\in{\mathbb{G}}$,

$\left(\nabla^{{\mathbb{G}}}_{E_{k}}E_{\ell}\right)_{Q}=\frac{1}{2}\left[E_{k},E_{\ell}\right]^{{\mathbb{G}}}_{Q}=\frac{Q}{2}{\llbracket}\epsilon_{k},\epsilon_{\ell}{\rrbracket}.$
(4.2)

Now, in order to derive $(\ref{LccO2})$, decompose $U$ and $V$ as
$U=\sum\limits_{k=1}^{N}u_{k}E_{k}$ and
$V=\sum\limits_{\ell=1}^{N}v_{\ell}E_{\ell}$. Then, apply the Leibniz rule and
conclude by $(\ref{LCClivf})$. ∎

######  Proposition 7.

Let $X,Y\in{\mathcal{X}}({\mathbb{B}})$ and $b\in{\mathbb{B}}$. Then, for any
$Q\in\pi^{-1}(b)$,

$\left(\nabla^{{\mathbb{B}}}_{X}Y\right)_{b}=\left(d\pi\right)_{Q}\left(Q\left(d\overline{{Y^{\sharp}}}\right)_{Q}\left({X^{\sharp}}(Q)\right)\right)+\frac{1}{2}\left(d\pi\right)_{Q}\left(Q{\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right).$
(4.3)

Suppose that Assumption 1 holds. Then,

$\left(\nabla^{{\mathbb{B}}}_{X}Y\right)_{b}=\left(d\pi\right)_{Q}\left(Q\left(d\overline{{Y^{\sharp}}}\right)_{Q}\left({X^{\sharp}}(Q)\right)\right).$
(4.4)

###### Proof.

Since $\pi$ is a Riemannian submersion, we have that
$\left(\nabla^{B}_{X}Y\right)_{b}=\left(d\pi\right)_{Q}\left(\left(\nabla^{{\mathbb{G}}}_{{X^{\sharp}}}{Y^{\sharp}}\right)_{Q}\right)$.
So $(\ref{eqLCCofB})$ follows from $(\ref{LccO2})$. For the second part,
Assumption 1 implies that for all $Q\in{\mathbb{G}}$,
${\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\in\mathrm{Ver}^{\pi}_{e}$.
Now, by Lemma 1, for all $Q\in{\mathbb{G}}$,
$Q{\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\in\mathrm{Ver}^{\pi}_{Q}$.
∎

###  4.3 Curvature

Throughout the sequel, we assume that $\left\\{\epsilon_{k}:1\leq k\leq
N\right\\}$ is an orthonormal basis of ${\mathfrak{g}}$ such that
$\left\\{\epsilon_{k}:1\leq k\leq N^{h}\right\\}$ is an orthonormal basis of
$H^{\pi}_{e}$ and $\left\\{\epsilon_{k}:N^{h}+1\leq k\leq N\right\\}$ is an
orthonormal basis of $V^{\pi}_{e}$. Since $g^{{\mathbb{G}}}$ is a left-
invariant metric, for all $Q\in{\mathbb{G}}$, $\left\\{E_{k}(Q):1\leq k\leq
N^{h}\right\\}$ and $\left\\{E_{k}(Q):N^{h}+1\leq k\leq N\right\\}$ are
respectively orthonormal basis of $H^{\pi}_{Q}$ and $V^{\pi}_{Q}$.

####  4.3.1 $(0,4)$ curvature tensor

######  Lemma 4.

For any $X,Y,Z,W\in{\mathcal{X}}({\mathbb{B}})$ and $Q\in{\mathbb{G}}$,

$R^{{\mathbb{G}}}_{Q}\left({X^{\sharp}},{Y^{\sharp}},{Z^{\sharp}},{W^{\sharp}}\right)=\frac{1}{4}g_{e}^{{\mathbb{G}}}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket},{\llbracket}\overline{{Z^{\sharp}}}(Q),\overline{{W^{\sharp}}}(Q){\rrbracket}\right).$

###### Proof.

∎

######  Lemma 5.

For any $X,Y\in{\mathcal{X}}({\mathbb{B}})$ and $Q\in{\mathbb{G}}$,

$\mathrm{ver}_{Q}^{\pi}\left(\left[{X^{\sharp}},{Y^{\sharp}}\right]^{{\mathbb{G}}}_{Q}\right)=Q\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right).$

###### Proof.

For any $U,V\in{\mathcal{X}}({\mathbb{G}})$,

$\left[U,V\right]^{{\mathbb{G}}}=\sum\limits_{k,\ell=1}^{N}\left[u_{k}E_{k},v_{\ell}E_{\ell}\right]^{{\mathbb{G}}}=\sum\limits_{k,\ell=1}^{N}u_{k}(E_{k}\cdot
v_{\ell})E_{\ell}+u_{\ell}(E_{\ell}\cdot
v_{k})E_{k}+u_{k}v_{\ell}\left[E_{k},E_{\ell}\right]^{{\mathbb{G}}}.$

Since ${X^{\sharp}}$ and ${Y^{\sharp}}$ are horizontal vector fields, the
choice of the basis $\left\\{\epsilon_{k}:1\leq k\leq N\right\\}$ of
${\mathfrak{g}}$ implies that

$\left[{X^{\sharp}},{Y^{\sharp}}\right]^{{\mathbb{G}}}=\sum\limits_{k,\ell=1}^{N^{h}}x^{\sharp}_{k}(E_{k}\cdot
y^{\sharp}_{\ell})E_{\ell}+x^{\sharp}_{\ell}(E_{\ell}\cdot
y^{\sharp}_{k})E_{k}+x^{\sharp}_{k}y^{\sharp}_{\ell}\left[E_{k},E_{\ell}\right]^{{\mathbb{G}}}.$

So, for any $Q\in{\mathbb{G}}$,

$\mathrm{ver}_{Q}^{\pi}\left(\left[{X^{\sharp}},{Y^{\sharp}}\right]^{{\mathbb{G}}}_{Q}\right)=\sum\limits_{k,\ell=1}^{N^{h}}x^{\sharp}_{k}(Q)y^{\sharp}_{\ell}(Q)\mathrm{ver}_{Q}^{\pi}\left(\left[E_{k},E_{\ell}\right]^{{\mathbb{G}}}_{Q}\right).$

Now, by Lemma 1,

$\mathrm{ver}_{Q}^{\pi}\left(\left[E_{k},E_{\ell}\right]^{{\mathbb{G}}}_{Q}\right)=\mathrm{ver}_{Q}^{\pi}\left(Q{\llbracket}\epsilon_{k},\epsilon_{\ell}{\rrbracket}\right)=Q\mathrm{ver}_{e}^{\pi}\left({\llbracket}\epsilon_{k},\epsilon_{\ell}{\rrbracket}\right).$

Therefore,

$\mathrm{ver}_{Q}^{\pi}\left(\left[{X^{\sharp}},{Y^{\sharp}}\right]^{{\mathbb{G}}}_{Q}\right)=Q\sum\limits_{k,\ell=1}^{N^{h}}x^{\sharp}_{k}(Q)y^{\sharp}_{\ell}(Q)\mathrm{ver}_{e}^{\pi}\left({\llbracket}\epsilon_{k},\epsilon_{\ell}{\rrbracket}\right)=Q\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right).$

∎

######  Proposition 8.

For any $X,Y,Z,W\in{\mathcal{X}}({\mathbb{B}})$, set

${\mathfrak{R}}^{{\mathbb{B}}}(X,Y,Z,W):=g^{{\mathbb{G}}}\left(\mathrm{ver}^{\pi}\left(\left[{X^{\sharp}},{Y^{\sharp}}\right]^{{\mathbb{G}}}\right),\mathrm{ver}^{\pi}\left(\left[{Z^{\sharp}},{W^{\sharp}}\right]^{{\mathbb{G}}}\right)\right).$

Then, for all $b\in{\mathbb{B}}$ and any $Q\in\pi^{-1}(b)$,

$\displaystyle
R^{{\mathbb{B}}}_{b}\left(X,Y,Z,W\right)=\frac{1}{4}g_{e}^{{\mathbb{G}}}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket},{\llbracket}\overline{{Z^{\sharp}}}(Q),\overline{{W^{\sharp}}}(Q){\rrbracket}\right)+\frac{1}{2}{\mathfrak{R}}^{{\mathbb{B}}}_{b}(X,Y,Z,W)-\frac{1}{4}{\mathfrak{R}}^{{\mathbb{B}}}_{b}(Y,Z,X,W)-\frac{1}{4}{\mathfrak{R}}^{{\mathbb{B}}}_{b}(Z,X,Y,W).$

where

${\mathfrak{R}}^{{\mathbb{B}}}_{b}(X,Y,Z,W):=g_{e}^{{\mathbb{G}}}\left(\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right),\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{Z^{\sharp}}}(Q),\overline{{W^{\sharp}}}(Q){\rrbracket}\right)\right).$

###### Proof.

By O’Neill’s formula,

$R^{{\mathbb{B}}}\left(X,Y,Z,W\right)=R^{{\mathbb{G}}}\left({X^{\sharp}},{Y^{\sharp}},{Z^{\sharp}},{W^{\sharp}}\right)+\frac{1}{2}{\mathfrak{R}}^{{\mathbb{B}}}(X,Y,Z,W)-\frac{1}{4}{\mathfrak{R}}^{{\mathbb{B}}}(Y,Z,X,W)-\frac{1}{4}{\mathfrak{R}}^{{\mathbb{B}}}(Z,X,Y,W).$
(4.5)

The first term of $(\ref{ON4})$ is provided by Lemma 4. By Lemma 5, the other
terms of $(\ref{ON4})$ are given by

${\mathfrak{R}}^{{\mathbb{B}}}_{b}(X,Y,Z,W)=g_{e}^{{\mathbb{G}}}\left(\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right),\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{Z^{\sharp}}}(Q),\overline{{W^{\sharp}}}(Q){\rrbracket}\right)\right).$

∎

####  4.3.2 $(1,3)$ curvature tensor

Recall that $\left\\{\epsilon_{k}:1\leq k\leq N^{h}\right\\}$ is an
orthonormal basis of $H_{e}^{\pi}{\mathbb{G}}$. Now, for any
$b\in{\mathbb{B}}$ and fixed $Q\in\pi^{-1}(b)$, set

$w_{k}(b):=(d\pi)_{Q}(Q\epsilon_{k})\in T_{b}{\mathbb{B}}.$

Since the metric $g^{{\mathbb{G}}}$ is left-invariant and $\pi$ is a
Riemannian submersion, $\left\\{w_{k}(b):1\leq k\leq N^{h}\right\\}$ is an
orthonormal basis of $T_{b}{\mathbb{B}}$. So, for the $(3,1)$ curvature tensor
$R^{{\mathbb{B}}}\left(X,Y\right)Z$, for any $b\in{\mathbb{B}}$,

$\displaystyle R^{{\mathbb{B}}}_{b}\left(X,Y\right)Z$
$\displaystyle=\sum\limits_{k=1}^{N^{h}}g^{{\mathbb{B}}}_{b}\left(R^{{\mathbb{B}}}_{b}\left(X,Y\right)Z,w_{k}(b)\right)w_{k}(b)$
$\displaystyle=\sum\limits_{k=1}^{N^{h}}R^{{\mathbb{B}}}_{b}\left(X,Y,Z,W_{k}\right)w_{k}(b),$

where $W_{k}$ is any vector field such that $(W_{k})_{b}=w_{k}(b)$. For any
$b\in{\mathbb{B}}$ and fixed $Q\in\pi^{-1}(b)$, Proposition 8 provide the
value at $b$ of the tensor $R^{{\mathbb{B}}}$ in function of that of the
vector fields:

$R^{{\mathbb{B}}}_{b}\left(X,Y,Z,W_{k}\right)=\rho_{b}\left(X_{b},Y_{b},Z_{b},(W_{k})_{b}\right).$

Therefore,

$R^{{\mathbb{B}}}_{b}\left(X,Y\right)Z=\sum\limits_{k=1}^{N^{h}}\rho_{b}\left(X_{b},Y_{b},Z_{b},w_{k}(b)\right)w_{k}(b).$

####  4.3.3 Sectional curvature

######  Proposition 9.

Let $X,Y\in{\mathcal{X}}({\mathbb{B}})$ be orthonormal vector fields. Then,
for all $b\in{\mathbb{B}}$ and $Q\in\pi^{-1}(b)$,

$K^{{\mathbb{B}}}_{b}(X,Y)=\frac{1}{4}\left\|{\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right\|^{2}_{e}+\frac{3}{4}\left\|\mathrm{ver}_{e}^{\pi}\left({\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right)\right\|^{2}_{e}.$
(4.6)

Suppose that Assumption 1 holds. Then, for all $b\in{\mathbb{B}}$ and
$Q\in\pi^{-1}(b)$,

$K^{{\mathbb{B}}}_{b}(X,Y)=\left\|{\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}\right\|^{2}_{e}.$
(4.7)

###### Proof.

By O’Neill’s formula,

$K^{{\mathbb{B}}}_{b}(X,Y)=K^{{\mathbb{G}}}_{P}(X,Y)+\frac{3}{4}\left\|\mathrm{ver}_{Q}^{\pi}\left(\left[{X^{\sharp}},{Y^{\sharp}}\right]^{{\mathbb{G}}}_{Q}\right)\right\|^{2}_{Q}.$

Then, Lemmas 5 and 4 combined to the left-invariance of the metric
$g^{{\mathbb{G}}}$ imply $(\ref{eqSectCurvature})$, and
$(\ref{eqSectCurvatureSimple})$ follows readily. ∎

###  4.4 The case of Grassmannians and flag manifolds

First, notice that the Frobenius metric is bi-invariant, so that the metric
considered on $O(d)$ is.

Let $X,Y\in{\mathcal{X}}({\mathcal{F}^{\mathrm{I}}})$. For any
${\mathcal{P}}\in{\mathcal{F}^{\mathrm{I}}}$ and
$Q\in({\pi^{\mathrm{I}}})^{-1}({\mathcal{P}})$,

${\llbracket}\overline{{X^{\sharp}}}(Q),\overline{{Y^{\sharp}}}(Q){\rrbracket}=\frac{1}{4}Q^{T}\Omega_{{\mathcal{P}}}^{X,Y}Q,$

where

$\Omega_{{\mathcal{P}}}^{X,Y}:=\left(\sum\limits_{i,j}{\llbracket}{\llbracket}X_{i}({\mathcal{P}}),P_{i}{\rrbracket},{\llbracket}Y_{j}({\mathcal{P}}),P_{j}{\rrbracket}{\rrbracket}\right)\in{\mathfrak{so}(n)}.$

######  Lemma 6.

Assumption 1 holds for ${G_{n}^{q}}$ but not for ${\mathcal{F}^{\mathrm{I}}}$.

###### Proof.

This follows from simple computations in ${\mathfrak{so}(n)}$. ∎

####  4.4.1 Levi-Civita connection of Grassmannians and flag manifolds

######  Proposition 10.

For any $X,Y\in{\mathcal{X}}({G_{n}^{q}})$ and $P\in{G_{n}^{q}}$,
$Q\in({\pi^{m}})^{-1}(P)$,

$\left(\nabla^{{G_{n}^{q}}}_{X}Y\right)_{P}=\left(d{\pi^{m}}\right)_{Q}\left(Q\left(d\overline{{Y^{\sharp}}}\right)_{Q}\left({X^{\sharp}}(Q)\right)\right),$

where ${X^{\sharp}}(Q)={\llbracket}X({\pi^{m}}(Q)),{\pi^{m}}(Q){\rrbracket}Q$
and
$\overline{Y^{\sharp}}(Q)=Q^{T}{\llbracket}Y({\pi^{m}}(Q)),{\pi^{m}}(Q){\rrbracket}Q\in{\mathfrak{so}(n)}$.

###### Proof.

Since Assumption 1 holds for ${G_{n}^{q}}$, the result follows readily from
$(\ref{eqLCCofBsimple})$. ∎

We obtain an expression of the latter when $X$ and $Y$ are defined through
charts of ${G_{n}^{q}}$.

######  Proposition 11.

For any $X,Y\in{\mathcal{X}}({\mathcal{F}^{\mathrm{I}}})$ and
${\mathcal{P}}\in{\mathcal{F}^{\mathrm{I}}}$,
$Q\in\left(\pi^{{\mathrm{I}}}\right)^{-1}({\mathcal{P}})$,

$\left(\nabla^{{\mathcal{F}^{\mathrm{I}}}}_{X}Y\right)_{{\mathcal{P}}}=\left(d\pi^{{\mathrm{I}}}\right)_{Q}\left(Q\left(d\overline{{Y^{\sharp}}}\right)_{Q}\left({X^{\sharp}}(Q)\right)\right)+\frac{1}{8}(d{\pi^{\mathrm{I}}})_{Q}\left(\Omega_{{\mathcal{P}}}^{X,Y}Q\right).$

where
$X^{\sharp}(Q)=\frac{1}{2}\left(\sum\limits_{i=1}^{r}{\llbracket}X_{i}({\pi^{\mathrm{I}}}(Q)),{\pi^{\mathrm{I}}}(Q){\rrbracket}\right)Q$
and
$\overline{Y^{\sharp}}(Q)=\frac{1}{2}Q^{T}\left(\sum\limits_{i=1}^{r}{\llbracket}Y_{i}({\pi^{\mathrm{I}}}(Q)),{\pi^{\mathrm{I}}}(Q){\rrbracket}\right)Q\in{\mathfrak{so}(n)}$.

###### Proof.

Apply Proposition 7. ∎

####  4.4.2 Curvature of Grassmannians and flag manifolds

This general framework allows us to recover the sectional curvature of
Grassmannians, given in Proposition 4.1. of Bendokat Zimmermann and Absil 2020
(Bendokat Zimmermann and Absil, 2020).

######  Proposition 12.

Let $X,Y\in{\mathcal{X}}({G_{n}^{q}})$ be orthonormal vector fields. Then, for
all $P\in{G_{n}^{q}}$ and $Q\in\pi^{-1}(P)$,

$K^{{G_{n}^{q}}}_{P}(X,Y)=\left\|{\llbracket}X_{P},Y_{P}{\rrbracket}\right\|^{2}_{e}.$

###### Proof.

First, notice that the statement makes sense. Indeed, for all
$P\in{G_{n}^{q}}$, ${\llbracket}X_{P},Y_{P}{\rrbracket}\in\mathfrak{so}(n)$.
Now, the result follows from $(\ref{eqSectCurvatureSimple})$, since Assumption
1 holds for Grassmannians. ∎

######  Proposition 13.

Let $X,Y\in{\mathcal{X}}({\mathcal{F}^{\mathrm{I}}})$. Then, for all
${\mathcal{P}}\in{\mathcal{F}^{\mathrm{I}}}$,

$K^{{\mathcal{F}^{\mathrm{I}}}}_{{\mathcal{P}}}(X,Y)=\frac{1}{16}\left(\left\|\Omega_{{\mathcal{P}}}^{X,Y}\right\|^{2}_{e}+3\left\|\mathrm{ver}_{e}^{{\pi^{\mathrm{I}}}}\left(\Omega_{{\mathcal{P}}}^{X,Y}\right)\right\|^{2}_{e}\right),$

where for any $\Omega\in{\mathfrak{so}(n)}$,

$\mathrm{ver}_{e}^{{\pi^{\mathrm{I}}}}\left(\Omega\right)=\Omega-\frac{1}{2}\sum\limits_{i=1}^{r}{\llbracket}{\llbracket}\Omega,P_{0}^{i}{\rrbracket},P_{0}^{i}{\rrbracket}.$

###### Proof.

This is a consequence of $(\ref{eqSectCurvature})$. ∎

### Acknowledgements

The authors have received funding from the European Research Council (ERC)
under the European Union’s Horizon 2020 research and innovation program (grant
agreement G-Statistics No 786854). It was also supported by the French
government through the 3IA Côte d’Azur Investments ANR-19-P3IA-0002 managed by
the National Research Agency.

## References

  * [1] Alekseevsky, D.V. (1996).  Flag manifolds.  Yugoslav Geometrical Seminar Divcibare, 10-17 October 1996.
  * [2] Batzies, E., Huper, K., Machado, L. and Silva Leite, F. (2015).  Geometric mean and geodesic regresssion on Grassmannians.  Linea Algebra Appl., 466:83-101, 2015.
  * [3] Bendokat, T., Zimmermann, R., Absil, P.A. (2020).  A Grassmann Manifold Handbook: Basic Geometry and Computational Aspects.  arXiv preprint arXiv:2011.13699, 2020.
  * [4] Draper, B. (2014).  A flag representation for finite collections of subspaces of mixed dimensions.  Linear Algebra and its Applications, 15-32, 2014.
  * [5] Grove, K. (2000).  Geometry of and via symmetries, in Conformal, Riemannian and Lagrangian.  The 2000 Barrett Lectures, University Lecture Series (AMS), vol. 27., 2000.
  * [6] Guijarro, L. and Walschap, G. (2007).  When is a Riemannian submersion homogeneous?.  Geometria Dedicata 125, pp. 47-52, 2007.
  * [7] Kerin, M. and Shankar, K. 2012 Guijarro, L. and Walschap, G. (2007).  Riemannian submersions from simple, compact Lie groups.  Münster Journal of Mathematics, 5 (1). pp. 25-40, 2012.
  * [8] Michor, P.W. (2008).  Topics in Differential Geometry.  Volume 93 of Graduate Studies in Mathematics. American Mathematical Soc., 2008.
  * [9] Monk, D. (1959).  The geometry of flag manifolds.  Proc. London Math. Soc. (3), 9:253-286, 1959.
  * [10] Nguyen, D. (2021).  Closed-form geodesics and trust-region method to calculate Riemannian logarithms on Stiefel and its quotient manifolds.  arXiv:2102.13327, 2021.
  * [11] Nishimor, Y. Akaho, S. and Plumbley, M. (2006).  Riemannian optimization method on generalized flag manifolds for complex and subspace.  AIP Conference Proceedings, 89-96, 2006.
  * [12] O’Neill, B. (1966).  The fundamental equations of a submersion.  Michigan Math. J. 13(4): 459-469, 1966.
  * [13] Pennec, X. (2017).  Sample-limited L p Barycentric Subspace Analysis on Constant Curvature Spaces.  Geometric Science of Information, Springer International Publishing, 20-28, 2017.
  * [14] Pennec, X. (2018).  Barycentric subspace analysis on manifolds.  Ann. Statist. 46 (6A) 2711 - 2746, December 2018.
  * [15] Ye, K., Wong, K.S.W., Lim, L.H. (2022).  Optimization on flag manifolds.  Mathematical Programming (Series A), 2022, 194(1-2), pp. 621-660.